Enforcing Engineering Standards in AI-Assisted Development

The Problem: AI That Ships Fast But Not Well

Six months ago, I was building out a side project—implementing features, refactoring code, the usual dance of iterative development. I'd been using AI coding tools for a while: ChatGPT is my daily go to for everything, and I mainly use Claude Code when I want to build something. No matter which tool I used, or which model chose to complete tasks there was something that just didn't feel right to me, quality.

I noticed things I wouldn't expect from an experienced developer. I understood why it was happening: I didn't ask for these things specifically, and that's the root cause. Instead of building one reusable component, it would duplicate the same UI across multiple views with slight variations. Authentication got wired directly to Supabase's API without any abstraction layer I could swap out later. Worst of all, when debugging issues, it would sprinkle console.log statements throughout the codebase like confetti, then never clean them up.

I recognised these patterns immediately. This was code that worked but lacked the long-term thinking that comes from maintaining large projects: functional, but without considering the bigger picture of maintainability. The AI was implementing things "in a way it has been trained," which was understandable but not always right for the situation.

What I was seeing aligned perfectly with what's being called "vibe coding": people building products based on feel and intuition rather than engineering principles. You see the stories in the news: AI-generated apps that work in demos but fall apart in production, security breaches from systems with no proper authentication layers, performance issues from architectures that don't scale.

The pattern is always the same: builders assume certain fundamentals will be handled automatically and are oblivious to core principles of software development. They focus on features and user experience (which is important) but skip the infrastructure that makes software maintainable and reliable. The AI doesn't enforce good practices by default, so if you don't know to ask for them, you don't get them. Getting an application out and running is one thing, but when things go wrong (and they will) do you have the tools to find and fix the bug? Decent logging, observability, testing, error handling?

Don't get me wrong: I think it's fantastic that people are building stuff. Having ideas is usually the hardest part, and the drive to get them out into the world is exactly what innovation needs. In many ways, the AI is failing these builders by just doing what's asked without highlighting the gaps. It's like having a builder who builds exactly what you specify but doesn't mention that you forgot to plan for plumbing or electrical work. The intent is good, but the guidance is missing.

Good developers know instinctively how to implement something in the best way for the situation. AI tends to just implement it in a way that works. The difference is enormous.

The Vision: Enforcing Standards Through Process

I didn't want to build a workaround or accept lower standards. I wanted the AI I was using to be smarter and hold itself to higher standards. If it could understand complex codebases well enough to implement features, it should be capable of implementing them properly—with proper error handling, testing, architectural consistency, and all the other practices that make software sustainable.

What I've built isn't groundbreaking—many developers have implemented sub-agent workflows, and the concept has been around for a while. But Claude Code's introduction of officially supported sub-agents has made this approach much more accessible and reliable.

The solution I eventually built addresses this through what I call a "staged agent pipeline": seven specialised AI agents that each handle one piece of the development workflow, with sophisticated memory management and token budgeting to keep everything efficient and consistent.

The Architecture: Seven Agents, One Brain

The pipeline works like this:

User Prompt → BA_REFINE → ARCH_CHECK → DESIGN_SPEC → TEST_PLAN → IMPLEMENT_SCAFFOLD → REVIEW → DOCS → Working Code

Each stage is a separate Claude Code invocation with its own specialized prompt, budget, and success criteria. The key technical insight is that I'm using Claude Code's official sub-agent system rather than building custom orchestration from scratch:

# From run-auto.sh
claude "Use the auto-pipeline sub agent to: $*"

This leverages Claude Code's native sub-agent support, making the approach much more accessible since it uses official features rather than complex custom tooling.

Stage 1: BA_REFINE - The Requirements Engineer

The BA_REFINE agent takes my prompts ("the task edit modal isn't saving due to a 500 error") and transforms them into structured requirements with acceptance criteria. It reads through related code to understand the domain context, then produces a detailed analysis document.

## Problem Statement
Task edit modal submission returns 500 error on save operation

## Acceptance Criteria
- Given: User edits task details in modal
- When: User clicks "Save Changes"  
- Then: Task updates successfully with 200 response
- And: Modal closes with success feedback
- And: UI reflects updated task data

What makes this stage crucial is that it forces the AI to be specific about success criteria before any code gets written. Too often, I'd see later stages go off on tangents because the original intent wasn't clearly defined.

Stage 2: ARCH_CHECK - The System Architect

This agent maps the requirements onto the actual codebase architecture. It identifies which modules, services, and database tables are involved, flags any potential architectural issues, and creates a change impact analysis.

Instead of scanning the entire codebase (which would blow the token budget), ARCH_CHECK uses a pre-built index file that lists all source files with metadata. It then selectively reads only the files likely to be relevant.

# Generated by build-index.sh
{"path":"backend/src/modules/tasks/tasks.controller.ts","lines":300,"modified":"2025-01-15"}
{"path":"backend/src/modules/tasks/tasks.service.ts","lines":150,"modified":"2025-01-10"}

This stage also enforces what I call "hard exclusions": directories like node_modules, dist, and build that should never be analysed. Early versions of the pipeline would occasionally get stuck scanning dependency files, burning through tokens without useful results.

Stage 3: DESIGN_SPEC - The UI/UX Guardian

For frontend changes, DESIGN_SPEC ensures consistency with the design system. It checks that any UI modifications follow established patterns, use the right Tailwind classes, and maintain accessibility standards.

This agent reads the design system documentation and existing component library to understand established patterns. It's surprisingly good at catching when a quick fix would introduce visual inconsistencies or accessibility regressions.

Stage 4: TEST_PLAN - The Quality Engineer

TEST_PLAN generates the testing strategy and writes skeleton test cases that need to pass for the implementation to be considered complete. At the moment, it's mainly unit tests, but I'm looking to expand it to integration tests, end-to-end scenarios, and even performance benchmarks where relevant.

The key insight here is writing tests before implementation, which forces the AI to think through edge cases and error conditions upfront. It also gives the later REVIEW stage concrete criteria for evaluating the implementation.

Stage 5: IMPLEMENT_SCAFFOLD - The Coder

This stage takes all the previous analysis and actually modifies the codebase. It reads the relevant source files, applies the planned changes, writes new tests, and updates configuration files as needed.

The implementation includes automatic compilation verification as a quality gate:

# From run-auto.sh - pipeline fails if code doesn't compile
if npm run build >/dev/null 2>&1; then
    echo "✅ Backend compiles successfully"
else
    echo "❌ Backend compilation failed!"
    exit 1
fi

This ensures that no broken code makes it through the pipeline, providing an immediate feedback loop for implementation quality.

Stage 6: REVIEW - The Senior Developer

REVIEW acts like a senior engineer doing a code review. It examines the implementation against the original requirements, checks for security issues, validates architectural decisions, and verifies that tests adequately cover the changes.

This stage has three possible verdicts:

• APPROVE: Changes are good to go
• CONDITIONAL_GO: Changes work but has minor issues to address later
• BLOCK: Critical problems that must be fixed

Only APPROVE and CONDITIONAL_GO trigger the final DOCS stage.

Stage 7: DOCS - The Technical Writer

The final agent updates documentation, README files, API references, and changelog entries to reflect the changes. It also appends a structured summary to our persistent history file for future context, along with lessons learned that get stored in dedicated lesson files for future reference.

The Memory System: Context That Persists

The biggest technical challenge was providing Claude with rich context about the project from the beginning, rather than forcing it to rediscover everything in each conversation. When Claude starts fresh every time, it has to spend tokens and time exploring the codebase architecture, understanding domain patterns, and figuring out established conventions that I'd already settled on in previous work.

My solution has two layers: a structured history log and a lessons learned system.

The main component is history.ndjson: an append-only log file where each line captures the essence of a completed pipeline run:

{
  "timestamp": "2025-01-15T14:30:00Z",
  "prompt": "Fix task edit modal save operation returning 500 error",
  "objectives": ["Debug PUT /tasks/:id endpoint", "Fix validation logic", "Add error handling"],
  "changed_files": ["tasks.controller.ts", "tasks.service.ts", "update-task.dto.ts"],
  "review_verdict": "APPROVE",
  "summary": "Fixed missing validation in DTO. Added proper error responses for invalid task IDs.",
  "git_commit": "a7b8c9d",
  "tags": ["bugfix", "backend", "validation"]
}

Beyond the main history log, I also maintain lesson files (like task-edit-500-fix-lessons.md) that capture specific insights and patterns discovered during implementation. This creates a layered memory system where the JSON provides quick context while lesson files offer deeper architectural and domain insights.

The beauty of NDJSON (newline-delimited JSON) is that it's trivial to parse and append to. Each run just adds one line. No complex database schema or file locking issues.

Token Economics: The Hidden Costs of AI Development

Before building the pipeline, the efficiency problem hit me hard when I started running into Claude Code's usage limits after just three hours of development work. I'd be in flow, implementing features and refactoring code, then suddenly get locked out for several hours until my quota reset. The same thing was happening with Cursor: I was burning through tokens faster than I could work.

After implementing my first version of the pipeline, I started playing detective to refine it. I'd copy entire terminal outputs and feed them to ChatGPT, asking it to add up the token usage and give me a general breakdown of where they were being spent across the different stages: BA_REFINE, ARCH_CHECK, and so on. What I discovered was that a significant portion of tokens was being spent reanalysing files that had already been read in previous conversations. Claude was essentially doing the same exploratory work over and over again because it had no context about what it had already learnt about my codebase.

I also noticed that the self-reported token usage from the agents didn't match what the CLI was showing. This made it difficult to understand where the actual inefficiencies were and how to optimize the process.

This wasn't just about hitting limits. Token efficiency directly impacts the quality of AI assistance. When the AI wastes tokens on irrelevant files, it has less context budget for understanding the actual problem and implementing a good solution.

The budget allocation itself reflects the relative complexity of each stage:

• IMPLEMENT_SCAFFOLD: 35k tokens (34% of total) Gets the largest allocation since it's doing actual code modification and compilation verification
• ARCH_CHECK/TEST_PLAN: 15k each (15% each) Need substantial context for analysis and planning
• BA_REFINE/DOCS: 8k each (8% each) Can be more efficient since they focus on structured output rather than deep code analysis

To track actual usage, I built a simple token counting system that parses CLI output. This lets me track actual token usage patterns and tune the budgets over time. While the self-reported numbers from agents often don't match CLI reality, the tracking helps identify when stages are being inefficient.

The Exclusion System: Teaching AI What Not to Read

Speaking of node_modules: one of the most frustrating early bugs was watching the pipeline burn through thousands of tokens scanning dependency directories. The AI would see a task related to "authentication" and decide it needed to understand every library in node_modules/.

The solution was a hard exclusion system. Most of this is now handled by Claude Code's settings.json file, but I keep explicit stage instructions as a safety net:

HARD EXCLUSIONS (never list, open, or count):
- **/node_modules/**
- **/dist/** **/build/** **/.next/**
- **/coverage/** **/.turbo/** **/.git/**
- **/.cache/** **/.eslintcache**

I also built a pre-processing script that generates a clean file index excluding these directories, giving the AI a clean view of the actual application code without the noise of dependencies and build artifacts.

Sub-Agent Orchestration and Quality Gates

The technical implementation relies on Claude Code's official sub-agent system rather than complex custom orchestration. The key integration point uses a custom hook:

# Custom hook that triggers the pipeline
/auto this is the prompt

# Which internally calls:
claude "Use the auto-pipeline sub agent to: $*"

This leverages Claude Code's native capabilities while adding quality gates around the process. Beyond the compilation verification mentioned earlier, the pipeline includes several other checks:

Test Coverage Maintenance: The backend maintains comprehensive test coverage reporting in /coverage, ensuring the pipeline doesn't just write tests but maintains testing quality.

Production Integration: The implementation includes real production concerns like Honeycomb observability integration (honeycomb.service.ts) and sophisticated test infrastructure with mocks (supabase.mock.ts, honeycomb.mock.ts) and fixtures (task.fixtures.ts).

Database Migration Handling: The system includes sophisticated migration management with rollback scripts, operating in a real production context rather than toy examples.

Real Results: What Success Looks Like

Here's an actual entry from history.ndjson showing a successful pipeline run:

{
  "timestamp": "2025-01-15T14:30:00Z",
  "prompt": "when trying to edit a task, the modal opens, I edit what I need and hit Save Changes. However, nothing saves. 500 error in console",
  "review_verdict": "CONDITIONAL_GO", 
  "stages": {
    "BA_REFINE": {"used": 8200, "exceeded": false},
    "ARCH_CHECK": {"used": 15100, "exceeded": false}, 
    "DESIGN_SPEC": {"used": 7800, "exceeded": false},
    "TEST_PLAN": {"used": 14900, "exceeded": false},
    "IMPLEMENT_SCAFFOLD": {"used": 34800, "exceeded": false},
    "REVIEW": {"used": 12400, "exceeded": false},
    "DOCS": {"used": 8100, "exceeded": false}
  },
  "changed_files": ["tasks.controller.ts", "tasks.service.ts", "update-task.dto.ts", "task-edit.component.ts"],
  "git_commit": "a7b8c9d",
  "summary": "Fixed validation error in task update DTO. Added proper error handling and user feedback.",
  "tags": ["bugfix", "validation", "frontend", "backend"]
}

This run consumed 101.3k tokens total, with IMPLEMENT_SCAFFOLD using 34% of the budget as expected. The "CONDITIONAL_GO" verdict indicates the fix worked but had minor issues to address in future iterations.

The beauty of this system is that future runs can reference this context: "Previously fixed similar validation issues in task updates (commit a7b8c9d), apply similar patterns to user profile updates."

The Prompt Problem: Specificity Is Everything

Learning to work with the pipeline taught me that prompt quality makes or breaks the entire workflow. The difference between a good prompt and a bad one isn't subtle—it's the difference between getting exactly what you need and getting something that technically works but isn't useful. It's like the difference between saying "make me a sandwich" and saying "can I have a ham salad sandwich, on granary, no onions please." The specificity isn't pedantic—it's essential.

Bad prompt: "Build user authentication feature"
Good prompt: "Add JWT-based session authentication to the tasks API. Users should be able to login with email/password, get a token that expires in 24 hours, and all task CRUD operations should require valid authentication. Include proper error responses for invalid credentials and expired tokens."

This applies especially to the BA_REFINE stage, which sets the tone for everything that follows. If BA_REFINE gets vague requirements, every subsequent stage compounds that vagueness until you end up with something that works but isn't what you actually needed.

I don't use formal templates or user story formats: there are plenty of examples of those that have been around for years. I just try to be specific, as if I was working with someone to implement something. The key is thinking through not just what you want, but what good implementation looks like: error handling, edge cases, integration points, testing scenarios.

But getting to this level of automation required solving numerous edge cases and failure modes:

Budget Exceeded Scenarios: When stages hit their token limits, the pipeline logs this in the orchestrator progress and adjusts subsequent stages accordingly rather than failing completely.

Compilation Failures: The automatic build verification catches broken code immediately, providing fast feedback without waiting for manual testing.

Directory Confusion: Early versions would sometimes cd into subdirectories and create artefacts in the wrong places. I had to add explicit working directory rules and absolute path resolution.

Context Window Overflow: Stages trying to read too many files at once. Solved with the bounded exploration strategy and file size limits.

What Works and What Doesn't

After many iterations, I've learned where this approach shines and where it still struggles.

Where it excels:

• Feature implementation on existing architectures
• Bug fixes with clear symptoms (like the 500 error example)
• Refactoring existing code to improve structure or performance
• Adding testing, logging, and observability to existing features

Where it still needs help:

• Initial architecture decisions for greenfield projects
• Complex UI design without visual references (Claude Code can't see screenshots like Cursor or GitHub Copilot)
• Integration with completely new external services
• Performance optimisation requiring deep system knowledge

The system works best once you have something up and running. I like to get a proof of concept built in a bare-bones way, then use the pipeline to build on that foundation once I've done an architectural review.

The success rate has a lot to do with prompt specificity. Generic requests get generic implementations. Specific, well-thought-out prompts that include context about existing patterns and constraints consistently produce good results.

What I'd Do Differently

If I were starting over, I'd make a few changes:

Parallel Execution: Some stages could run in parallel. DESIGN_SPEC and TEST_PLAN don't actually depend on each other and could execute simultaneously.

Smarter Context Loading: Instead of injecting the same history summary into every stage, I'd customise the context based on what each stage actually needs.

Better Error Recovery: When a stage fails, the entire pipeline stops. I'd add retry logic and graceful degradation for non-critical failures.

Real Token Tracking: The self-reported token numbers are often inaccurate. I'd build more sophisticated token tracking into the orchestration layer from day one, though the current basic tracking has been sufficient for identifying major inefficiencies.

Incremental Updates: Currently, each run overwrites the previous artefacts. I'd add versioning so I could track how requirements and implementations evolve over time.

The Bigger Picture: Engineering Discipline in an AI World

What I've built isn't groundbreaking—many developers have implemented sub-agent workflows, and the concept has been around for a while. But Claude Code's introduction of officially supported sub-agents has made this approach much more accessible and reliable.

The real value isn't in the technical implementation—it's in addressing a fundamental problem with AI-assisted development. When AI makes it easy to ship code quickly, it becomes easy to skip the disciplinary practices that make software maintainable.

This has huge implications beyond just my personal workflow. A lot of people coming into software development through AI tools have fantastic product ideas but no background in software engineering. They're going to miss the important foundational steps—proper error handling, logging, observability, testing—and the consequences are enormous.

The pipeline enforces these practices automatically. Every implementation includes proper error handling. Every feature gets tests. Every change maintains architectural consistency. It's like having a senior developer's checklist built into the AI's execution process.

This becomes critical when you're moving fast on side projects or early-stage products. The temptation is always to cut corners and "add proper testing later." But later never comes, and you end up with a prototype that can't evolve into a real product.

My current side project—a system that helps non-technical founders maintain engineering standards while building their products—has been the perfect testing ground. The pipeline has handled everything from complex data modeling to API design to frontend state management. When things do go wrong, I have the observability and logging infrastructure to diagnose issues quickly.

For me, this is about getting ideas out and making them useable while maintaining the standards I'd expect from any development team. The AI doesn't replace good engineering judgment—it automates the execution of that judgment.