Your AI agent just shipped broken code. Again. Here's the pattern most teams are missing.
The Problem Nobody Talks About
We've all seen the demos. An AI agent takes a task description, writes the code, runs the tests, opens a PR. Magic.
Then a human reviews the PR and leaves comments: "This breaks the existing API contract." "You forgot to handle the edge case we discussed." "Wrong architectural pattern for this module."
The agent fixes the PR. Ships it. Everyone moves on.
But here's the thing — next time, the agent makes the exact same mistakes. Different task, same class of errors. It didn't learn anything from the review.
This is the missing feedback loop.
Why Agents Keep Repeating Mistakes
Most AI agent setups treat each task as a blank slate. The agent gets a prompt, does the work, and forgets everything. It's like hiring a contractor who has amnesia between jobs.
The core issue: there's no mechanism to feed review feedback back into the agent's decision-making process for future tasks.
Think about how humans learn on a team:
- Junior dev writes code
- Senior dev reviews it: "We don't do X here, we do Y because Z"
- Junior dev remembers this for next time
- Over weeks, the junior dev internalizes the team's patterns
AI agents are stuck permanently at step 1 and 2. Step 3 never happens.
The Feedback Loop Pattern
After months of running autonomous coding agents in production, here's the pattern that actually works:
1. Capture Review Feedback as Structured Lessons
When a human reviewer leaves comments on an agent's PR, don't just fix the code. Extract the lesson:
- What the agent did: Generated a REST endpoint with positional arguments
- What was wrong: Team convention is keyword-only arguments for all public APIs
- The rule: All public API functions must use keyword-only arguments (
*separator)
Store these as structured entries, not free-form text. You need to be able to retrieve them later by context.
2. Inject Relevant Lessons Before Each Task
Before the agent starts a new task, query your lesson store for anything relevant to:
- The module being modified
- The type of work (API design, database migration, UI component, etc.)
- The specific technologies involved
Inject these as part of the agent's context. Not as a massive dump of every lesson ever learned — that kills performance. As a targeted selection of the 5-10 most relevant lessons for this specific task.
3. Track Recurrence
Monitor whether the same type of mistake keeps appearing. If an agent keeps making the same error despite having the lesson in context, you have a deeper problem — maybe the lesson is poorly worded, maybe the context window is too large and the lesson gets lost, or maybe the underlying model just can't reliably follow that particular instruction.
4. Graduate Lessons to System Prompts
Once a lesson has proven itself (agents consistently follow it when it's in context), promote it from per-task injection to the agent's base prompt or system instructions. This reduces retrieval overhead and ensures the rule is always present.
What This Looks Like in Practice
In my setup, I run coding agents that pick up tasks from a project management system, implement them, and submit merge requests. Early on, the pattern was painful:
- Agent writes code → human reviews → 3-4 rounds of feedback → finally merges
- Next task, same module → agent makes the same architectural mistakes
- Human reviewer gets frustrated: "I told you this last time"
After implementing the feedback loop:
- Review comments get extracted into a lessons database
- Before each task, the agent receives relevant lessons as additional context
- First-time mistakes still happen (that's fine), but repeat mistakes dropped by roughly 70%
- Review rounds went from 3-4 to typically 1-2
The key insight: the lessons database becomes the team's institutional knowledge, encoded in a format agents can actually use.
The Hard Parts
Lesson quality matters enormously. A vague lesson like "write better tests" is useless. A specific one like "when modifying the payment flow, always add a test for the webhook retry scenario" is gold.
Context window management is tricky. You can't dump 500 lessons into every prompt. You need good retrieval — semantic search, tagging by module/technology, relevance scoring.
Human buy-in is non-negotiable. Someone has to review the extracted lessons for accuracy. An agent that learns the wrong lesson from a misunderstood review comment will confidently make new, creative mistakes.
Lesson decay is real. Codebases evolve. A lesson about the old API pattern becomes actively harmful when the team migrates to a new pattern. You need a process to retire outdated lessons.
The Bigger Picture
This isn't just about code quality. It's about the fundamental question of how we work with AI agents over time.
Right now, most teams treat AI agents as stateless tools. Use them, get output, forget. But the teams getting the most value are treating them as team members who need onboarding, feedback, and institutional knowledge — just delivered in a different format.
The feedback loop is the minimum viable version of this. It's how you go from "AI agent that writes code" to "AI agent that writes code the way your team writes code."
And honestly? Building this loop taught me as much about how humans learn on teams as it did about AI. The patterns are remarkably similar. We just never had to make them explicit before.
Omar Díaz Peña builds autonomous development systems and writes about the messy reality of AI in production.