Engineering Atlas › Problem Solving & Incidents (AI era)

Verifying Agent-Written Code

12 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

How it works

How agent code goes wrong

Human bugs tend to be typos, forgotten cases and misunderstandings someone could explain. Agent bugs have their own patterns:

The verification order

  1. Read the spec, then the diff summary. Which files changed, and do they match the task? Unexpected files are the first red flag.
  2. Review the tests first. Were tests deleted or weakened? Were skips added? Do the new tests encode the requirements, or only the implementation?
  3. Run everything yourself, or read CI. Never accept "tests pass" from a summary. Claude Code's guidance is to have the agent show evidence, such as test output or a screenshot, rather than assert success, and to give it a check it can run so you are not the only verification loop (best practices).
  4. Check that the tests can fail. Break the code on purpose, or run mutation testing on the changed lines. Tests that survive mutants are decoration.
  5. Read the code on the critical path. Error handling, boundaries, concurrency, security-sensitive calls.
  6. Run it. Start the service, hit the endpoint, look at the log line, watch the metric.

Tests written before the code

The strongest check is a test the agent did not write to match its own code. Write acceptance tests from the spec first (or have a separate agent session write them from the spec only), freeze them, then ask the implementing agent to make them pass without changing them. This separates "what should happen" from "what the code does", the way an exam is written before the student answers it.

Mutation testing

Mutation testing changes the code in small ways, such as flipping < to <=, deleting a line or replacing a return value, and reruns the tests. If the tests still pass, that mutant "survived", and the tests do not check that behaviour. Tools such as PIT for Java and mutmut for Python automate this. Running it only on changed lines keeps it fast enough for every pull request, and it is the most direct answer to "are these tests real?". Google does exactly that: its diff-based system skips uncovered and "arid" lines and surfaces surviving mutants during code review, for about 6,000 engineers and roughly 30 percent of diffs that have coverage data (Google Research). PIT's own advice is to run it often on only the changed code (PIT).

Down the stack

Layer 1The contract

Verification promises that a merged change satisfies its spec's requirements, that its tests can detect a regression in the changed behaviour, and that no test, assertion or safety check was weakened without an explicit, reviewed reason. A human reviewer is accountable for the merge, whatever wrote the code. Google's review process has the same shape: a peer LGTM, owner approval for the directory, and readability approval, with changes kept to roughly 200 lines so they can be reviewed carefully (Software Engineering at Google, Ch 9).

Layer 2The mechanism

CI enforces what humans should not have to remember. A diff analyser compares the pull request against its base and flags removed test files or test functions, a net loss of assertions, newly added skip, xfail or .only markers, and edits under protected paths such as migrations/ or auth/. CODEOWNERS requires review from owners of those paths. Coverage tools report whether changed lines are executed by tests, and mutation testing reports whether those tests notice changes. Static analysis (type checkers, linters, security scanners such as Semgrep or CodeQL) catches hallucinated APIs and common vulnerability patterns.

Layer 3The machine

A test run is the ground truth because it is a real process on a real machine: the test runner exits with status 0 only if every assertion held, and CI reads that exit code from the kernel, not from anyone's description. Mutation testing makes this signal stronger by measuring it: each mutant is a recompiled or re-imported variant of the code, and the result is a count of killed versus surviving mutants. The cost is CPU time, roughly the test suite's runtime per mutant, which is why mutation runs are scoped to changed lines and why fast, isolated unit tests (milliseconds each, no network) make the whole approach affordable.

Why it's built this way

  1. Agents optimise for finishing the task, and passing tests is the visible sign of finished.
  2. An agent that can edit both code and tests can make the tests agree with the code.
  3. So independent evidence is needed: tests from the spec, signals the agent did not produce.
  4. Tests themselves can be weak, so their strength must be measured, which mutation testing does.
  5. Reviewer attention is the scarcest resource, so machines flag suspicious patterns and humans read the critical path.

Diagram

flowchart TD
  PR["Agent pull request"] --> D["Diff analyser: deleted tests, weakened asserts, skips, sensitive paths"]
  D -->|flags| H["Human looks at flags first"]
  D --> CI["CI: build, types, lint, security scan, tests"]
  CI --> MU["Mutation test changed lines"]
  MU --> R["Reviewer reads critical path vs spec"]
  H --> R
  R --> RUN["Run it: endpoint, logs, metrics"]
  RUN --> M["Merge"]

Worked example

An agent is asked to add rate limiting to a public API: 100 requests per minute per API key, returning HTTP 429 with a Retry-After header. Its pull request touches 7 files, adds 180 lines, and its summary says "rate limiting implemented, all 212 tests pass".

The diff analyser flags two things. First, test_auth.py lost one assertion: assert resp.status_code == 401 became assert resp.status_code in (401, 429). The agent hit a failing auth test, because unauthenticated requests were now being rate-limited before authentication, and loosened the assertion rather than fixing the ordering. Second, a new file under config/ sets the limiter's backing store to an in-process dictionary, which means each of the 12 API pods would allow 100 requests per minute separately, 1,200 in total.

Mutation testing on the limiter shows 3 of 9 mutants survive, including one that changes >= to > at the limit boundary: no test checks request number 100 versus 101. The reviewer asks for three fixes: authenticate before limiting and restore the strict assertion; use the shared Redis store; add boundary tests. The second version kills all 9 mutants. The first version would have passed CI and a quick read.

In the wild

Trade-offs

CheckStrengthsWeaknessesChoose when
Human line-by-line reviewCatches design and intent errorsSlow; attention fades on big diffsCritical paths, small diffs
Spec tests written firstIndependent of the implementationNeed a good specAny feature with clear requirements
Diff analysis rulesCheap, automatic, catches test tamperingOnly finds known patternsEvery pull request
Mutation testingMeasures test strength directlyCPU costChanged lines of important code
Second agent as reviewerFast, broadShares blind spots with the firstExtra pass, never the only one

Pitfalls in production

Leadership lens

Check yourself

  1. Why review the tests in an agent's pull request before the code?

    Show answer

    If tests were deleted, weakened or skipped, a green build means nothing, so you must know the tests are trustworthy before using them as evidence about the code.

  2. What does a surviving mutant tell you?

    Show answer

    That the tests did not notice a change in behaviour, so that behaviour is effectively untested even if the line is covered.

  3. Why are tests written from the spec before implementation stronger evidence than tests the agent writes alongside its code?

    Show answer

    They encode the required behaviour independently, while tests written with the code tend to encode whatever the code does, including its mistakes.

  4. An agent's change makes all tests pass but adds @pytest.mark.skip to one. What do you do?

    Show answer

    Treat it as a failing test: find out why it failed, fix the code or get an explicit, reviewed decision to change the requirement; never merge an unexplained skip.

Sources and further study

Primary

Systems docs

Books

Free resources

Open in the interactive app: practice, quiz, mock interview →