Code is cheap now. Review is the bottleneck.
TL;DR: Writing code got cheap with AI, and I think checking it did not. In my plan, work in progress gets a cap, changes stay small, authors bring proof and each change goes down a path that fits its risk. People keep the merge, and machines prove the rest.
Imagine cooks who suddenly work several times faster, so dishes stack up at the window while one person tastes everything with a single spoon. In my picture that taster is code review. A smaller taster who inspects the whole table at the end is QA.
A working paper from Harvard tracked 718 firms. Once agents arrived, they wrote about 30% more lines of code, yet each PR took 49% longer to merge, and resolved issues did not clearly rise. Nobody has checked the paper yet the way journals do, so for my purposes it is a hint, not proof.
In my reading, vendor numbers show things moving together, not one causing the other. Faros AI gathered data on over 10,000 developers. Teams leaning on AI merged 98% more PRs, almost twice as many. Their reviews took 91% longer, and the typical PR grew 154%. LinearB says an agent’s PR waits 5.3x longer before anyone looks at it. That is for the slowest quarter of PRs.
Nearly 5,000 people took part in the survey behind the 2025 DORA report. It ties AI to more output and shakier delivery.
Meta’s RADAR paper says landed diffs about doubled in length within a year. Developers shipped 51% more of them per month, and over 80% of that growth was put down to agentic AI. Fewer got reviewed within a day, but no number is given.
Another team looked at 33,596 agent PRs, and 61% showed no recorded review. Careful, though. It only counts reviews that left a trace, and human PRs in those repos look almost the same. A third study checked 33,707 agent PRs, and over a quarter merged instantly.
A growing queue makes it worse. Each new plate gets a faster, sloppier tasting. First the taster samples everything, then skims, then just nods. In my eyes, the review has become a rubber stamp by then.
Ten extra tasters are not in my budget, and I don’t trust a robot to taste alone. So I pull four levers: fewer plates, quicker checks, slow tasting only where the risk is, and machines proving what they can.
One rule before any trick
That brings me to my one hard rule. No AI approves or merges, ever. People hold the rules, the limits and the merge button. A bot may flag a risky chunk, sketch a checklist or write a test. I see that as a tip for the person deciding, never as the decision.
RADAR is Meta’s system for reviewing the safe-looking diffs without a person. Over 535K diffs went through it and over 331K landed, while Meta’s teams choose the thresholds and who is eligible, set caps, keep a denylist and ask for a 60-day clean record. Experts review incidents, but spot checks aren’t described.
Some RADAR diffs do land with no human approval. A bot’s diff lands after a delay, and a human may reject it in that window. For a human’s diff, the RADAR approval replaces human review. That is looser than my rule.
Reverts hit RADAR’s diffs at a third of the usual rate, and incidents at one fiftieth, but the authors say this doesn’t prove cause, because those diffs were low-risk from the start. So it’s no proof that automatic review is safe. My own line is stricter: a human keeps the merge, even for tiny changes. That is my taste, not a finding.
Stop cooking when the pass is full
My favourite fix is the bluntest one: a cap on work in progress. Picture a pass with twelve spots. Once twelve plates wait there, the cooks stop.
Little’s law backs this up: the queue length equals the arrival speed times the time an item waits, though it holds only as an average in a calm system. For my purposes it is a rough guide. Suppose a team gets through 12 PRs daily and wants each one reviewed inside a day. Twelve is the most its queue should hold.
In my design the gate sits before the agents, not at the PR. A branch that is finished but unopened is a plate hiding in the back, and that is worse than a queue you can see, because nobody knows how many plates are waiting. It stops my agents when the queue is full:
#!/usr/bin/env bash
# dispatch-gate.sh: call it first in whatever script launches an agent
set -euo pipefail
CAP="${REVIEW_CAP:-12}"
open=$(gh pr list --repo "$REPO" --state open --label needs-review \
--search "draft:false" --limit 100 --json number --jq length)
if [ "$open" -ge "$CAP" ]; then
echo "Review queue is full ($open of $CAP). Review first, then dispatch." >&2
exit 1
fiMy plan also fixes two hours a day for review, so the queue has somewhere to drain.
The cost is real. Folks will feel stuck on day one, and some will say the queue is someone else’s job. The cap exists to break that habit.
Smaller plates
In a reviewer’s list, a 50-file diff looks just like a 3-line auth fix. So I’d start with a size budget in CI, which is one portion per plate. About 500 changed lines sounds right to me, minus lockfiles and generated code:
name: pr-size
on: pull_request
jobs:
size:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- run: |
# holds only with the default merge-ref checkout (no ref: head.sha)
changed=$(git diff --numstat HEAD^1 HEAD \
-- . ':!*.lock' ':!*lock.json' ':(exclude,glob)**/generated/**' \
| awk '{s += $1 + $2} END {print s + 0}')
echo "Changed lines: $changed (budget: 500)"
test "$changed" -le 500A diff over the budget fails the pipeline with one message: split it. My agent would get a standing order: deliver small PRs that each pass alone, and send refactors first, then behaviour changes, then cleanup. I wouldn’t raise the limit to suit the AI, because a higher limit only hides the root problem.
The cook brings the proof
Whoever directed the AI knows what they wanted, so I’d make them bring the proof in an evidence block in our PR template. It’s a cook sending a plate out with a note: what is in it, what I tasted, what might go wrong.
## Evidence
- What changed and why:
- What I ran (tests, static analysis, mutation score, environment URL):
- Risks I see:
- How to revert:
- AI-generated? Which parts I read line by line:In my pipeline a small CI job rejects any PR without the block, so the block goes on top of the template:
name: evidence-block
on:
pull_request:
types: [opened, edited, synchronize, reopened]
jobs:
evidence:
runs-on: ubuntu-latest
env:
BODY: ${{ github.event.pull_request.body }}
steps:
- run: echo "$BODY" | grep -q '^## Evidence' || { echo "Fill in the Evidence block."; exit 1; }My runner-up is a trick I call explain to merge. A bot looks at the diff and sends the author three short questions. Why this approach? What breaks when a value is null? What if two writes arrive at once? The author replies in the PR, and only then does a reviewer begin.
I think this costs around five minutes. Now the reviewer sees which parts the author grasped and which they only hoped were fine. A weak answer means a coaching chat, not a blocked merge. The bot only asks. A person reads the replies, since a bot marking its own homework is just a rubber stamp.
Taste by risk, not by arrival
I’d split reviews into three lanes: the light lane, the standard lane and the deep lane.
| Lane | What goes in | Humans involved |
|---|---|---|
| Light | Under the size budget, evidence block complete, all machine gates green. A proven pure refactor, or a dependency bump with passing contract tests | The author merges after one quick acknowledgement, and a sampled audit follows |
| Standard | Medium risk | One reviewer, one business day |
| Deep | Auth, payments, migrations, data deletion, public contracts, tests and CI | Two reviewers plus QA, a short call, a feature flag |
The lane rules sit in a repo file, so changing them takes an PR with human review, and a classifier job reads the changed paths to pick the lane. In my plan it launches in shadow mode. Think of a trainee who jots a verdict on a sticky note and may not send a plate back. It prints a lane and a reason, and changes nothing. Reviewers compare it with their gut feeling, and I’d let it route only after they trust it.
# holds only with the default merge-ref checkout (no ref: head.sha)
files=$(git diff --name-only HEAD^1 HEAD)
ORACLE='^(tests/|\.github/workflows/|phpstan\.neon|phpunit\.xml)'
DEEP='^(src/Auth/|src/Billing/|migrations/|api/openapi\.yaml)'
# oracle-change quarantine: tests and CI never travel with production code
if echo "$files" | grep -Eq "$ORACLE" && echo "$files" | grep -Evq "$ORACLE"; then
echo "Oracle change mixed with production code. Split the PR." >&2
exit 1
fi
if echo "$files" | grep -Eq "$ORACLE|$DEEP"; then lane=deep; else lane=standard; fi
echo "lane=$lane"That first check closes a gap that worries me. An agent that cannot make a test pass can quietly weaken the test, and I think code reading will not catch that reliably. The tests are the recipe card the taster checks against. Change the card and the dish together, and a bad dish passes.
So an PR that changes tests, fixtures, CI or lint rules goes to the deep lane and travels alone, with no production code in it. I call this oracle-change quarantine. An oracle is just whatever tells you the right answer.
Let machines prove what they can
Line coverage says a test ran. It can’t say the test is able to fail, but mutation testing can. Imagine a critic who swaps the salt for sugar and watches whether the taster notices. The tool makes small edits to your code, such as flipping a > to >=, and runs the tests again. If a test still passes, it missed the edit. In my plan, the gate is the mutation score of the changed files. PHP has Infection for that:
vendor/bin/infection --git-diff-lines --git-diff-base=origin/main \
--min-msi=70 --min-covered-msi=80 --threads=4Any edit that slipped past the tests goes back to the agent as a failing case. We repeat that until the score stops rising. My reviewer then reads the mutation report and skips most of the tests. I’m not sure 70 and 80 suit every project, so they are a first guess.
For behaviour, my favourite tool is a diff taken in an ephemeral environment. That is a pop-up kitchen, built for one night and torn down after. Run the old and the new build of the service side by side, then send both the same requests and compare:
mkdir -p out/old out/new
i=0
while read -r path; do
i=$((i + 1))
curl -s "$OLD_URL$path" | jq -S 'del(.updated_at)' > "out/old/$i.json"
curl -s "$NEW_URL$path" | jq -S 'del(.updated_at)' > "out/new/$i.json"
done < requests.txt
diff -ru out/old out/new > behaviour.diff || trueThis handles reads only. Anything that writes data needs more care, and setup costs climb. Even so, I prefer reading a short list of behaviour changes, each marked expected or not, to guessing from the source.
Trust the light lane, but check it
The light lane needs a checker, in my opinion, so I’d run an audit lottery. Weekly, a script would draw 10% of the light merges at random. A senior engineer then reviews them after the merge. The one question is whether they would have blocked it.
since=$(date -u -d '7 days ago' +%Y-%m-%d)
gh pr list --repo "$REPO" --state merged --label lane-light \
--search "merged:>=$since" --limit 100 --json url --jq '.[].url' > light.txt
shuf -n "$(( ($(wc -l < light.txt) + 9) / 10 ))" light.txtBefore the start, agree how many escapes are too many. When the audits cross that number, that kind of change loses the light lane. In my plan, a switch like that keeps lighter review defensible. I’d open my light lane the day the audit begins and never before.
Where QA fits
QA, to me, goes at the front. Before anyone writes code, QA and the developer settle what done means, as acceptance criteria in the ticket. The AI turns those into tests. It is like fixing the menu before lighting any stove. Each PR gets a small table: criterion, test, evidence. I’d read the table first and open the code only where a row is empty.
QA effort then follows the lanes. Light gets automated regression only. Standard gets a short look at the risky spots. For the deep lane I’d write a charter and run a timed session of exploring. AI can turn a charter into test cases. Still, I’d leave the question of what could go wrong to people.
What I’d do first
In my plan, the first step is two weeks of measuring time in review, queue length and PR size. Next I’d add the cheap items in one go: gate, size budget, template. After that I’d add machine gates, the shadow-mode risk score and finally the lanes. The audit runs from day one of the light lane. Ephemeral environments need the biggest setup, so I’d leave them for last.
First on my skip list is a second reviewer for every PR, since it doubles the wait. I’d also skip counting comments per reviewer, since that pays for noise. I’d never run an AI reviewer that comments on every line. People tune it out, and the one real finding drowns.
Which would you try first? Do you see holes in my plan?
