Engineering Atlas › Building AI Systems

Agent Architectures: Loops, Workflows and Sub-agents

12 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

How it works

The agent loop

  1. Send the goal, instructions, tools and the history so far to the model.
  2. The model returns either a final answer or one or more tool calls.
  3. Your code validates and executes the tool calls, records results.
  4. Append results to the history and go back to step 1.
  5. Stop when the model answers, a limit is hit, or a guard detects a problem.

The ReAct pattern (reason, act, observe), first published in October 2022 and presented at ICLR, popularised this loop; its authors reported that interleaving reasoning with actions beat imitation and reinforcement-learning baselines on ALFWorld and WebShop by absolute success rates of 34% and 10% (Yao et al.). Modern models are trained for it directly through tool use.

Workflows before agents

Anthropic's "Building effective agents" guidance draws a useful line: workflows are systems where your code orchestrates model calls along predefined paths; agents are systems where the model directs its own process. Both are built from an "augmented LLM", a model with retrieval, tools and memory, and the guidance advises finding the simplest solution and adding complexity only when needed (Anthropic). Microsoft's Azure Architecture Center says the same in its own terms: use the lowest level of complexity that reliably meets your requirements, because each step up from a single call, to a single agent with tools, to multiagent orchestration, adds coordination overhead, latency and cost (Azure Architecture Center). Common workflow patterns:

If you can draw the flowchart in advance, build a workflow. Agents earn their cost when the path depends on what is discovered along the way, such as debugging, research or multi-step support cases.

Limits and guards

Sub-agents

A lead agent can delegate subtasks to sub-agents that run with their own fresh context and return a summary. This keeps the lead's context clean and allows parallel work. Anthropic's write-up of its multi-agent research system (an Opus 4 lead with Sonnet 4 sub-agents) beat a single Opus 4 agent by 90.2% on its internal research eval, but agents use about 4 times the tokens of a chat and multi-agent systems about 15 times. Token usage alone explained 80% of the performance variance on BrowseComp, so part of the gain is simply more tokens spent (Anthropic). Sub-agents fit tasks with independent, parallel parts; the same post says most coding tasks have fewer parallelisable parts than research, and that work needing one shared context or many inter-agent dependencies is a poor fit. The early versions also misbehaved in instructive ways: they spawned dozens of sub-agents for simple queries and gave vague delegations that made sub-agents duplicate each other, which the team fixed with effort-scaling rules in the prompt (one agent with 3 to 10 tool calls for a simple lookup, 2 to 4 sub-agents for a comparison) and with delegations that state an objective, output format, tools and task boundaries.

State and durability

A long-running agent is a long-running job. Persist its state (goal, plan, history or a summary, completed tool calls with idempotency keys) so it can resume after a crash or deploy without repeating side effects. Treat each tool call like a step in a workflow engine. Anthropic's research system resumes from where an error occurred instead of restarting, combining retries with regular checkpoints, and its lead agent saves its plan to external memory because the context may be truncated past 200,000 tokens (Anthropic).

The same shapes as any system

Lilian Weng's survey splits an LLM-powered agent into planning, memory and tool use around the model as its "brain" (Weng). An agent loop is a retry loop with a planner: it needs the same budgets, timeouts, idempotency and stuck-detection as any job runner or saga. Sub-agents are fan-out with a merge step.

Down the stack

Layer 1The contract

An agent promises to pursue a goal within stated limits (steps, cost, time, permitted tools), to stop with an explicit reason, and to leave an auditable trace of every action. It does not promise success; it promises bounded, observable behaviour.

Layer 2The mechanism

The loop is ordinary code: a while loop around a model call, a tool dispatcher, validation, and an append to the message list. Frameworks and SDKs package this with features such as automatic context compaction when the history grows, permission checks per tool, hooks before and after tool calls, and sub-agent spawning. Loop detection compares recent tool calls by name and normalised arguments. Durable agents store each step in a database or a workflow engine (such as Temporal) so the loop can resume from the last completed step. Temporal does not snapshot memory: it reruns the workflow code against a recorded event history, reuses recorded activity results instead of re-running them, and therefore requires the workflow code to be deterministic, so model and tool calls belong in activities (Temporal docs).

Layer 3The machine

Each loop iteration resends the whole context, so cost grows with history: a 30-step agent whose context grows by 2,000 tokens per step processes roughly 30 times the average context size in input tokens, around a million tokens, unless caching or compaction intervenes. Prompt caching helps because the history is a stable prefix that grows at the end. Latency adds up the same way: 30 steps of 3 seconds each is 90 seconds of model time, plus tool time, which is why agents run as background jobs with progress updates rather than inside a single HTTP request.

Why it's built this way

  1. Some tasks cannot be planned in advance, so a model must choose the next step.
  2. Choosing steps in a loop risks wandering, looping and runaway cost.
  3. So loops need explicit limits, stuck detection and recorded stop reasons.
  4. When steps can be planned, workflows give the same results more cheaply and predictably.
  5. Long loops are jobs, so they need durable state and idempotent side effects.

Diagram

flowchart TD
  G["Goal and context"] --> M["Model call"]
  M --> D{"Final answer?"}
  D -->|yes| DONE["Stop: done"]
  D -->|no, tool call| V["Validate and authorise"]
  V --> T["Run tool (idempotent)"]
  T --> H["Append result to history"]
  H --> L{"Limits: steps, cost, time, repeats"}
  L -->|ok| M
  L -->|exceeded| STOP["Stop: budget or stuck"]

Worked example

A team builds a "triage this production alert" agent with tools for querying logs, metrics, recent deploys and the runbook. The first version is a free-form agent. In testing, 12 percent of runs hit 50 steps without an answer, many repeating the same log query, and the average run costs 1.10 dollars.

Looking at transcripts, most alerts follow the same path: check recent deploys, check error rates by service, read the runbook section for the alert, then summarise. The team turns that path into a workflow: three parallel tool calls (deploys, error rates, runbook), one model call to summarise and suggest next steps. Only when the summary says "cause unclear" does an agent take over, with a 15-step budget, loop detection (same tool and arguments three times stops the run) and a 0.50 dollar cap. Results on 200 past alerts: 78 percent handled by the workflow alone in about 6 seconds for 0.04 dollars; 22 percent escalate to the agent; no runs hit the budget; average cost per alert falls to 0.15 dollars; and on-call engineers rate the summaries as useful more often, because the workflow's output is consistent.

In the wild

Trade-offs

ApproachStrengthsWeaknessesChoose when
Single model callCheapest, fastestOne shot, no toolsSimple transformations
Workflow (chain, router, parallel)Predictable, testable, cheapFixed pathsSteps known in advance
Single agent loopHandles open-ended tasksCost, variance, harder to testPath depends on discoveries
Lead agent with sub-agentsParallel breadth, clean contextsMany more tokens, coordinationBroad research, independent subtasks
Workflow first, agent fallbackCheap common path, flexible tailTwo systems to maintainMixed routine and novel cases

Pitfalls in production

Leadership lens

Check yourself

  1. When should you build a workflow instead of an agent?

    Show answer

    When the steps can be known in advance; workflows are cheaper, faster and easier to test than letting a model choose each step.

  2. Why do long agent runs get expensive faster than linearly per step?

    Show answer

    Each step resends the growing history, so total input tokens grow roughly with the square of the number of steps unless caching or compaction reduces it.

  3. What does repeating the same tool call with the same arguments usually indicate?

    Show answer

    The agent is stuck in a loop and not making progress, so it should be stopped or escalated.

  4. When do sub-agents help, and what do they cost?

    Show answer

    They help with broad tasks that split into independent parallel parts and keep the lead context clean, but they use many more tokens and add coordination complexity.

Sources and further study

Primary

Systems docs

Books

Other curricula

Free resources

Open in the interactive app: practice, quiz, mock interview →