Engineering Atlas › Problem Solving & Incidents (AI era)

Whiteboard First: Judgment Before Automation

13 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

How it works

Why the order matters

The common workflow is: ask an AI for an idea, then use your own brain to tidy it up. It feels efficient, but it anchors you. The underlying effect is automation bias: the tendency to prefer an automated suggestion over other information, including correct information (Wikipedia). Once a plausible design is on screen, you edit it instead of questioning it, and your review measures how convincing the text is rather than whether the design is right. Research supports the worry. A 2025 Microsoft Research survey of 319 knowledge workers found that higher confidence in generative AI went with less critical thinking, while higher confidence in one's own ability went with more (Microsoft Research). In a Stanford experiment, people with an AI assistant wrote less secure code and were more likely to believe it was secure (Perry et al., 2022). A 2025 METR randomised trial gave 16 experienced developers 246 real issues from repositories they knew; with early-2025 tools they took 19 percent longer, after predicting a 24 percent speed-up, and afterwards still believed they had been about 20 percent faster (METR). It is a small study of old tools, and METR says newer data supersede it, so read it as evidence about the gap between felt and measured speed, not as a verdict on today's agents. The tools are useful; the trap is trusting the feeling of speed.

Reversing the order fixes the anchor. If you have drawn the boxes, named the invariants and written the decision yourself, the agent's output is checked against your frame, and gaps become visible.

The sequence rule

  1. Frame the problem in one sentence with a numeric success measure, with no AI open.
  2. Name the problem shapes present.
  3. Draw the components, data flow and critical path.
  4. List assumptions, marking each as known, estimated or guessed.
  5. Write the decision as an SDRC.
  6. Turn it into a spec: invariants become tests, shapes become mechanisms, risks become checks.
  7. Hand execution to the agent.
  8. Ask the agent to attack your plan, then verify its output against the spec.

Anthropic's own guidance for Claude Code has the same order: explore and plan first, then implement, and give the agent a check it can run. It also says to skip the plan when you could describe the diff in one sentence (Claude Code best practices). That is the zone idea in miniature.

A whiteboard is the best tool for steps 1 to 5 because it is slow, visual and shared: you cannot paste into it, and everyone in the room sees the gaps.

The four zones

Classify each task by stakes (what a mistake costs, and how hard it is to undo) and novelty (whether a known checklist exists or judgment is needed).

Stakes is the first question, because it decides whether a human must look at all. A migration script is routine, but a wrong one can delete data, so it is never "automate".

SDRC: proving ownership

SDRC is a one-page way to show you own a decision rather than relaying an AI's.

Writing the decision down before building is old practice: Google's design docs record goals, non-goals and the trade-offs considered, and a 1 to 3 page "mini design doc" suits incremental work (Ubl). The test is whether you can do it at a whiteboard without notes. AI is good at hiding uncertainty behind authoritative language; SDRC forces the uncertainty into the open, in the Risk and Change boxes.

REACT: rules for using AI on a task

Before AI touches a task, answer five questions: the Reason to use it (or not); the Evidence that will verify its output; who is Accountable for the result; the Constraints (legal, privacy, security, data it must not see); and the Trade-offs between speed and quality or understanding. Teams can turn these answers into a short policy per zone.

Down the stack

Layer 1The contract

The practice promises that for any high-stakes task, a human-authored frame exists before AI-generated work, a named person signs off, and the verification method was decided in advance. Each is auditable: the design doc or SDRC predates the agent session, the pull request names a reviewer, and the checklist is linked.

Layer 2The mechanism

Teams enforce zones with tooling, not memory. Labels on tickets or pull requests record the zone. Repository rules require code-owner review for paths in the gatekeeper zone, such as migrations/ or infra/iam/, using a CODEOWNERS file and a branch protection rule with "Require review from Code Owners" (GitHub Docs). Agent configuration files (CLAUDE.md, AGENTS.md) state the rules the agent should follow, for example "never run migrations; propose them in a PR". Treat those as advice: the agent reads them as context, not as enforced configuration, so a hard limit belongs in a permission rule or a hook (Claude Code memory docs). Hooks are deterministic where instructions are advisory (best practices). Permission settings allow or deny specific commands, so the automate zone can run freely while destructive commands need human approval. The SDRC lives in the design doc template, and the REACT answers in the ticket.

Layer 3The machine

Why a human must stay in the loop is partly about how language models produce text. A model generates one token at a time by sampling from a probability distribution, and the fluency of the result reflects how typical the wording is, not whether the claim is true. A confident sentence and a correct sentence look the same in the output. Models can report calibrated confidence in some settings, but a coding agent's prose summary ("all tests pass, migration is safe") is generated text, not a measurement. The only reliable signals are external ones: a test run's exit code, a type checker, a dry-run plan, a diff a human reads. Claude Code's docs say the same: without a check the agent can run, "looks done" is the only signal, and you become the verification loop (best practices). The whiteboard method turns those external checks into the definition of done.

Why it's built this way

  1. Fluent output triggers automation bias, so people under-check it.
  2. Under-checking is cheap for low-stakes work and very expensive for high-stakes work.
  3. So the amount of human judgment should depend on stakes and novelty, hence the zones.
  4. Judgment is anchored by whatever comes first, so the human frame must come before the AI draft.
  5. Frames are only useful if they are concrete and checkable, hence SDRC, REACT and specs as tests.

Diagram

flowchart TD
  T["New task"] --> S{"High stakes?"}
  S -->|yes| N1{"Novel?"}
  S -->|no| N2{"Novel?"}
  N1 -->|yes| A["Analog first: whiteboard, SDRC, then AI"]
  N1 -->|no| G["Human gatekeeper: AI drafts, checklist review"]
  N2 -->|yes| H["Human-shaped: AI options, human edits"]
  N2 -->|no| X["Automate: delegate, watch metrics"]

Worked example

A team must move user sessions from Postgres to Redis because the sessions table takes 40,000 writes a second and p99 login latency has reached 80 ms. It is high stakes (a mistake logs out every user or leaks sessions) and fairly novel for this team: analog first.

At the whiteboard, without AI, they write the SDRC. Situation: 12 million active sessions, 40,000 writes a second, 300,000 reads a second, sessions live 14 days, Postgres CPU at 85 percent. Decision: dual-write to Redis for 14 days, then switch reads, because a 14-day dual-write lets every live session exist in both stores without a bulk copy. Risk: Redis eviction could silently log users out if memory runs short; at about 1 KB per session, 12 million sessions need about 12 GB plus overhead, so they plan 32 GB with maxmemory-policy noeviction. Unknown: how many sessions are refreshed versus created. Change: roll back if login errors exceed 0.1 percent or Redis memory passes 75 percent.

Then the agent writes the dual-write code, the feature flag and the dashboards, against a spec with three tests from the invariants: no session readable in one store but not the other after a write; no eviction under load; reads switch per flag without restart. The pull request touches auth/, a gatekeeper path, so a named reviewer checks it against the SDRC. When the agent's summary claims "eviction is impossible", the reviewer checks the actual config instead, and finds the agent had set allkeys-lru. Ten minutes at the whiteboard caught a bug that would have logged out users at the first memory spike.

In the wild

Trade-offs

ApproachStrengthsWeaknessesChoose when
Analog firstHuman understanding; catches wrong framingSlower to startHigh stakes and novel
AI drafts, human gatekeepsFast drafting with a hard checkChecklist fatigue, rubber-stampingHigh stakes, routine
AI first, human editsMany options quicklyAnchoring on the first draftLow stakes, exploratory
Full automationMaximum speedSilent errors at scaleCheap, reversible, well-tested work

Pitfalls in production

Leadership lens

Check yourself

  1. Why is "AI first, then human cleanup" risky for high-stakes work even if the human is skilled?

    Show answer

    The first draft anchors judgment; the reviewer edits within its frame and checks how convincing it is rather than whether its framing is right, which is automation bias.

  2. A change adds an index to a 2 TB production table. Which zone, and why?

    Show answer

    Human gatekeeper: it is a routine, well-understood operation (low novelty), but a mistake can lock the table or exhaust disk (high stakes), so a human verifies it against a checklist.

  3. What does the Change box in SDRC add that the other boxes do not?

    Show answer

    Measurable triggers for reversing or revisiting the decision, which turns uncertainty into concrete monitoring and rollback criteria.

  4. Why are external signals better than an agent's own summary?

    Show answer

    The summary is generated text whose confidence reflects fluency, not truth; exit codes, tests, diffs and dry runs measure what actually happened.

Sources and further study

Primary

Systems docs

Books

Free resources

Open in the interactive app: practice, quiz, mock interview →