Engineering Atlas › Problem Solving & Incidents (AI era)

Context Engineering for Coding Agents

12 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

How it works

The context window is a budget

Every token in the window competes for the model's attention: system prompt, tool definitions, project memory files, the chat so far, every file read, every command output. Windows are large now, often 200,000 tokens or more, but research keeps finding that models use long contexts unevenly. The 2023 "Lost in the Middle" study found that models retrieve facts placed at the start or end of a long context far better than facts in the middle, even models built for long inputs (Liu et al.). Chroma's 2025 "context rot" tests of 18 models found performance on simple tasks declining as input grew, a single distractor lowering accuracy, and much better results on a roughly 300-token version of a task than on the same task padded to about 113,000 tokens (Chroma). Anthropic frames the same point as an attention budget: aim for the smallest set of high-signal tokens that gets the outcome (Anthropic). A context full of stale file contents and 3,000-line logs is a distracted context.

Four layers of context

Durable project memory. A file the agent reads at the start of every session: how to build and test, code style, architecture in a paragraph, things never to do ("never run migrations against prod", "use the db helper, not raw SQL"). Claude Code reads CLAUDE.md; many tools read AGENTS.md, an open format that the project says is used by more than 60,000 repositories (agents.md). Claude Code falls back to AGENTS.md only when no CLAUDE.md exists, and its docs suggest keeping each file under 200 lines because longer files cost context and reduce adherence (Claude Code memory docs). The best-practices guide adds a test for each line: would removing it cause mistakes? (best practices) Keep it short and specific; every line is paid for in every session.

Task context. The spec, the plan and the handful of files that matter for this task. Point the agent at them explicitly rather than hoping it finds them.

Fresh evidence. Test failures, stack traces, a log excerpt, a failing query plan. Trim it: the first error and its stack trace, not 4,000 lines of CI output.

Working memory. The conversation and tool results so far. This is the layer that rots. It grows with every read and retry and is full of dead ends.

Techniques that work

The same problem in other domains

Context engineering is a hot-spot and finding-things-fast problem in disguise: a scarce resource (attention) and a need to retrieve the right few items from many. It is the same skill as writing a good incident handover ("here is what we know, what we tried, what we suspect"), a good onboarding doc, or a good pull request description. People and models both work better with less, better-chosen information.

Down the stack

Layer 1The contract

A well-engineered context promises that the agent sees the project's rules and commands at session start, the spec and relevant files for the current task, and current evidence rather than stale copies. It also promises a budget: memory files under a few thousand tokens, tool outputs truncated, and sessions compacted or restarted before quality drops.

Layer 2The mechanism

Harnesses assemble context in layers. Claude Code, for example, loads CLAUDE.md files from the directories above the working directory at launch, loads a subdirectory's file only once it reads or edits a file there, and treats all of them as context rather than enforced settings (memory docs). Tool definitions and the conversation come on top. When the window nears its limit it compacts: older turns are replaced with a model-written summary. Retrieval happens through tools: grep, glob and file reads, or an index such as a code graph or embeddings. Sub-agents run with their own fresh context and return only a final message. Each design choice trades recall (did the agent see what mattered?) against focus (is it distracted?).

Layer 3The machine

Transformer attention compares every token with every other token in a layer, so compute for the attention step grows with the square of the sequence length. Production systems use tricks (key-value caches, sparse or sliding attention, prompt caching) to keep cost manageable, but the model still has to spread attention weights, which sum to 1 per query position, across everything present. Prompt caching also explains why stable content should come first: providers cache the processed prefix of a prompt. In Anthropic's API the prefix order is tools, then system, then messages; the default cache lifetime is 5 minutes, refreshed by each hit, and cached reads cost a fraction of normal input tokens. A change to tool definitions invalidates everything after them, and any change before a cache breakpoint breaks the match (prompt caching docs). So a fixed system prompt and memory file at the top are processed once and reused cheaply, while anything that changes invalidates the cache from that point onward. Put stable things first and volatile things last.

Why it's built this way

  1. Models attend to everything in the window, and attention is finite.
  2. Irrelevant tokens dilute attention and cost money and latency.
  3. So context must be selected: durable rules pinned, task material retrieved, evidence trimmed.
  4. Sessions accumulate noise, so they must be compacted, restarted or split into sub-agents.
  5. Stable content first also makes caching effective, so good context engineering is both more accurate and cheaper.

Diagram

flowchart TD
  M["Project memory: CLAUDE.md or AGENTS.md"] --> W["Context window"]
  S["Spec and plan"] --> W
  F["Retrieved files, on demand"] --> W
  E["Trimmed evidence: first failure"] --> W
  W --> A["Agent acts"]
  A -->|long or confused| C["Summarise and restart"]
  A -->|wide search| SA["Sub-agent explores, returns summary"]
  SA --> W
  C --> W

Worked example

An engineer asks an agent to fix a flaky checkout test in a 400,000-line monorepo. First attempt: they paste the whole CI log (6,000 lines, about 90,000 tokens), and the agent reads 30 files while exploring. After an hour, the context holds about 180,000 tokens; the agent starts re-reading files it already read and proposes a fix to a different test.

Second attempt, engineered: a fresh session. The repository's AGENTS.md (about 900 tokens) gives the test command and says "tests use the fake clock in testing/clock.py; never call time.sleep". The engineer gives the failing test's name, the first failure with its stack trace (60 lines), and the spec of the checkout timeout rule. The agent sends a sub-agent to find every use of real time in the checkout package; it returns a 200-word answer naming two places. Total main context: about 25,000 tokens. The agent finds that reserve_stock reads datetime.now() directly instead of the injected clock, fixes it, and runs the test 200 times in a loop to show it is stable. Twelve minutes, and the fix is right.

The model was the same. The context was different.

In the wild

Trade-offs

ApproachStrengthsWeaknessesChoose when
Paste everythingNothing is missedDistraction, cost, lost-in-the-middleTiny projects only
Retrieve on demand with toolsFocused, scales to big reposCan miss what it did not search forDefault for real codebases
Pack repo into one fileSimple, full pictureOnly works for small reposHanding a small project to a model once
Sub-agents for explorationKeeps main context cleanSummaries can drop detailsWide searches, research, log trawls
Frequent restarts with summaryFresh focusSummary may lose nuanceLong tasks, confused sessions

Pitfalls in production

Leadership lens

Check yourself

  1. Why can adding more files to the context make an agent worse?

    Show answer

    Attention is spread across everything present, irrelevant content distracts the model, and facts buried in the middle of long contexts are retrieved less reliably.

  2. What belongs in a project memory file, and what does not?

    Show answer

    Build and test commands, conventions, architecture in brief and hard rules belong there; long docs, task-specific details and anything that changes often do not, because every line costs tokens in every session.

  3. Why put stable content at the start of the prompt?

    Show answer

    Prompt caching reuses the processed prefix, so stable content first is computed once and reused, while any change invalidates the cache from that point.

  4. When should you use a sub-agent instead of reading files in the main session?

    Show answer

    For wide exploration whose details you will not need later, so the main context receives a short conclusion instead of thousands of tokens of reads.

Sources and further study

Primary

Systems docs

Books

Other curricula

Free resources

Open in the interactive app: practice, quiz, mock interview →