Engineering Atlas › System Design Fundamentals

Stopping cascading failures with a circuit breaker

11 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

How it works

Why timeouts alone are not enough

Suppose a dependency starts hanging and your client timeout is 2 seconds. At 500 requests per second, about 1,000 requests are in flight waiting on it at any moment. That exhausts thread pools and connection pools, so requests that do not even need that dependency start queueing. Retries make it worse. The caller fails slowly and expensively, which is how one sick service takes down its callers.

The state machine

In closed state, every call goes through and its outcome is recorded in a sliding window, either the last N calls or the last T seconds. When the window holds at least a minimum number of calls and the failure rate (errors plus timeouts, sometimes plus slow calls above a latency threshold) crosses a threshold such as 50%, the breaker opens.

In open state, calls fail immediately with a specific error, typically in microseconds. The caller uses a fallback: cached data, a default, a degraded feature, or a clear error to the user. After a wait duration, for example 30 seconds, the breaker moves to half-open.

In half-open state, a small number of trial calls are let through. Libraries differ on how they judge them. The classic description, which Azure follows, closes after a set number of consecutive successes and reopens on the first failure (Azure Architecture Center). Resilience4j waits for all permitted trial calls (10 by default), then compares their failure rate with the threshold, so one unlucky timeout does not reopen it (Resilience4j docs). Some implementations grow the wait exponentially on repeated failures.

Check library defaults before trusting them. Resilience4j ships with a 100-call window, a minimum of 100 calls, a 50% threshold, a 60-second open period and a 60-second slow-call threshold, which suits a busy service but reacts slowly on a quiet one. Netflix's Hystrix, which popularised the pattern, has been in maintenance mode since 2018; Netflix pointed new projects to Resilience4j and moved toward adaptive concurrency limits that react to measured latency instead of fixed thresholds (Hystrix README).

What counts as failure

Count timeouts, connection errors, and 5xx responses. Do not count 4xx client errors such as validation failures or 404s, since they say nothing about the dependency's health. A breaker that trips because users typed bad input is a self-inflicted outage.

Granularity

Use one breaker per dependency, and often per endpoint or per host. A single breaker for "the payments API" will open for every endpoint when one slow endpoint fails. Per-host breakers (outlier detection in proxies) eject one bad instance while healthy ones keep serving.

Alternatives to a binary breaker

A breaker is modal: it flips between "send everything" and "send almost nothing", and the open mode is rarely exercised. AWS avoids that by capping retries with a token bucket instead (AWS Builders' Library). Google uses client-side adaptive throttling: each client compares requests sent with requests the backend accepted over the last two minutes, and once requests exceed K times accepts (K = 2 is the suggested value) it rejects a growing fraction of new requests locally (Google SRE book). Both degrade smoothly in proportion to the failure rate.

Down the stack

Layer 1The contract

When closed, the breaker adds negligible overhead and passes every call through. When open, it rejects calls in microseconds with a distinguishable error and performs no network I/O. In half-open, it admits at most a configured number of trial calls. State transitions are driven only by observed outcomes and elapsed time, and are observable through metrics or events. It does not promise that a call allowed through will succeed, only that the caller will not keep spending resources on a dependency that is clearly failing.

Layer 2The mechanism

Resilience4j keeps outcomes in a sliding window, either COUNT_BASED (a ring of the last N results) or TIME_BASED (N one-second buckets of partial aggregates, so evaluating the failure rate is O(1) rather than a scan). Each record updates counters for failures and slow calls (above slowCallDurationThreshold). The state lives in an AtomicReference, and transitions use compare-and-set so concurrent threads agree on a single transition. Half-open is controlled by permittedNumberOfCallsInHalfOpenState. Resilience4j also offers FORCED_OPEN and DISABLED states for the manual override that operators need during an incident. Envoy expresses circuit breaking differently: per-cluster thresholds such as max_connections, max_pending_requests and max_requests (1,024 each by default) and max_retries (3) (Envoy docs), plus outlier detection that ejects individual hosts after 5 consecutive 5xx responses by default (Envoy docs).

Layer 3The machine

What a hung call holds is concrete. A Java platform thread reserves 1 MB of stack by default; 1,000 stuck threads tie up a gigabyte of address space plus scheduler work. Each call holds a socket, a file descriptor, and a connection-pool slot. Without an application timeout, a connect to an unresponsive host keeps retransmitting SYNs per tcp_syn_retries (6 by default on Linux, about 131 seconds in total on current kernels) (kernel docs), and a stalled established connection can wait on retransmission timers for many minutes. In contrast, the breaker's open-state check is an atomic read and a clock comparison, a few nanoseconds, and a CAS on transition costs tens of nanoseconds.

Why it's built this way

  1. Threads, sockets, and pool slots are finite, and a hung dependency consumes them at the request rate, so the caller must stop sending.
  2. A single failure says little about health, so decisions use a statistical window with a minimum sample size.
  3. Recovery time is unknown, so the breaker probes with a few trial calls instead of guessing.
  4. Many threads observe outcomes concurrently, so state changes use lock-free atomic transitions.

Diagram

stateDiagram-v2
    [*] --> Closed
    Closed --> Closed: success, or failure rate below threshold
    Closed --> Open: failure rate over threshold (min calls met)
    Open --> Open: calls rejected fast
    Open --> HalfOpen: wait duration elapsed
    HalfOpen --> Closed: trial calls succeed
    HalfOpen --> Open: any trial call fails

Worked example

A checkout service calls a fraud-scoring service at 800 requests per second with a 1-second timeout. Normally fraud scoring answers in 40 ms, so about 32 calls are in flight. Checkout has a pool of 200 worker threads.

One afternoon fraud scoring starts timing out. Without a breaker, in-flight calls climb to 800 (800 per second times 1 second), the thread pool is exhausted within a quarter of a second, and checkout's own p99 jumps to 10 seconds as requests queue. Checkout is effectively down even though it could approve low-risk orders without a score.

With a breaker configured as a 100-call window, 50% failure threshold, minimum 20 calls, 30-second open period, and 5 trial calls: the first timeouts are recorded one second into the incident, and at 800 calls per second the window collects its 50th failure about 60 ms later, so the breaker opens. Checkout suffers one to two seconds of queueing instead of the whole incident. From then on, fraud calls fail in microseconds and the fallback applies a rule: approve orders under $100 from customers with prior purchases, and hold the rest for asynchronous review. Thread usage stays near normal. Every 30 seconds, 5 trial calls test fraud scoring; 12 minutes later it recovers, trials succeed, and normal scoring resumes. Checkout lost no availability, and only 6% of orders were held.

In the wild

Trade-offs

OptionStrengthsWeaknessesChoose when
Timeouts onlySimple, always neededStill waits full timeout per call, pools fillLow-volume or non-critical calls
Retries with backoffRecovers from transient blipsAdds load to a failing dependencyBrief, independent failures
Circuit breakerFails fast, protects resources, enables fallbackTuning effort, possible flapping, false tripsHigh-volume calls to dependencies that can fail persistently
Load shedding at the calleeProtects the dependency itselfCallers still wait for rejectionsCombine with breakers on the caller side

Pitfalls in production

Leadership lens

Check yourself

  1. Why does a breaker need a minimum number of calls before it can trip?

    Show answer

    With few calls, one or two failures produce a high failure rate by chance, so the breaker would flap open on noise.

  2. Why should 4xx responses usually not count as failures?

    Show answer

    They indicate a problem with the request, not with the dependency's health, so counting them can open the breaker while the dependency is fine.

  3. What problem does the half-open state solve?

    Show answer

    It lets the caller test recovery with a few requests instead of either staying open forever or sending full traffic to a dependency that may still be failing.

Sources and further study

Primary

Systems docs

Books

Other curricula

Watch

Free resources

Open in the interactive app: practice, quiz, mock interview →