Engineering Atlas › Problem Solving & Incidents (AI era)

Framing the Problem Before Solving It

12 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

How it works

Symptom, problem, request

Three things often get mixed up. A symptom is what someone observed: "checkout feels slow". A problem is the gap between expected and actual behaviour, with numbers: "p95 checkout latency for EU mobile users is 3.1 s against a 1 s objective since 14:00 Tuesday". A request is someone's proposed fix: "please add more pods". Your first job, and Polya's first step in How to Solve It ("understand the problem", Wikipedia summary), is to turn the symptom into a problem and to hold the request at arm's length until the problem is clear. The request may be right. Often it treats a symptom.

The XY problem is the common failure: someone wants X, believes Y will get it, and asks for help with Y. The cure is one question: "What are you ultimately trying to achieve?" (xyproblem.info). Monitoring guidance draws the same line: a good signal says what is broken (the symptom) and, separately, why (a possibly intermediate cause), and one team's symptom is another team's cause (Google SRE book, ch. 6).

The problem statement template

Write this before opening an editor or a prompt. Five minutes is enough for most work.

IS and IS NOT

Borrowed from classic troubleshooting methods (Kepner and Tregoe made it famous in the 1960s), the contrast table asks four questions twice. What fails, and what similar thing doesn't? Where does it fail, and where doesn't it? When did it start, and when is it absent (time of day, after restarts)? How much: how many, how big, is it growing?

Every pair where the problem IS somewhere but IS NOT somewhere comparable is a clue. If the problem hits Android but not iOS, the cause lives in something that differs between them. If it began at 14:00 Tuesday and not before, look at what changed at 14:00 Tuesday. A cause must explain every IS and every IS NOT; any candidate that fails one entry is out.

Questions that reframe

Framing for an agent

An agent can't ask the product owner what they meant unless you build that into the loop. Give it the problem statement, the success condition as an executable check where possible, the constraints ("do not change the public API", "no new dependencies"), and the non-goals. Ask it to restate the problem and list its assumptions before writing code. Mismatches in that restatement are cheap to fix; mismatches in a 600-line diff are not.

Down the stack

Layer 1The contract

A framed problem promises a shared, falsifiable target. Two engineers reading it should agree on whether a proposed change meets it. Concretely: the "done when" condition can be evaluated by a query, test or dashboard; the scope says which population the metric is computed over; and the non-goals say what reviewers should not ask for. If two people would argue about whether the fix worked, the frame isn't finished.

Layer 2The mechanism

In production, the IS and IS NOT table is built by slicing telemetry along dimensions. Metrics systems store time series keyed by labels (region, app_version, endpoint, status); traces carry attributes on each span; logs carry structured fields. Framing a latency problem means querying something like histogram_quantile(0.95, sum by (le, region, app_version) (rate(http_request_duration_seconds_bucket[5m]))) and seeing which label values separate the bad series from the good ones. Tools such as Honeycomb's BubbleUp automate exactly this comparison: you select the anomalous region, and each field is charted for the selection against everything else, so fields whose distributions differ are the leads (Honeycomb docs). The limit is cardinality: each label value combination is a new series, so metrics usually lack high-cardinality fields like user ID, and traces or wide events are where the finest slicing happens.

Layer 3The machine

Numbers in a frame are only as good as where they were measured. A server-side latency histogram typically starts its timer when the application reads the request, which is after the kernel completed the TCP handshake and queued the connection in the listen backlog. On Linux the backlog argument to listen bounds fully established connections waiting for accept, is silently capped at net.core.somaxconn (128 before kernel 5.4, 4096 since), and when the queue is full the connection attempt may be refused or simply ignored so that a retry succeeds (listen(2)). The client then retransmits, and with no RTT sample yet the standard initial timeout is 1 second (RFC 6298); the server's histogram never sees that second. So "p95 is fine on the server, users say it's slow" is a framing clue pointing below the application: nstat counters such as TcpExtListenOverflows and TcpExtListenDrops, or ss -lnt showing a full Recv-Q on the listening socket. Similarly, Prometheus-style histograms estimate percentiles by interpolating inside a bucket, so with buckets at 0.5 s and 1 s a reported "p95 of 0.8 s" could be anywhere in that bucket; the docs state that the error is limited by the width of the bucket holding the quantile, and native histograms bound it by resolution instead (Prometheus docs). Check bucket edges before writing a precise target into a frame.

Why it's built this way

  1. People report symptoms through their own lens, so a frame has to translate a report into something measurable.
  2. Measurement is sliced by dimensions, so contrasts (IS versus IS NOT) are cheap to compute and narrow the search fast.
  3. Every measurement point sees only part of the path, so the frame must say where the number came from.
  4. Effort is expensive and agents amplify direction, so a few minutes of framing pays for itself many times.

Diagram

flowchart TD
    R["Report: 'uploads are broken'"] --> Q["Ask: expected vs actual, since when, who"]
    Q --> T["IS / IS NOT table"]
    T --> S["Slice telemetry by dimension"]
    S --> D{"Which values separate bad from good?"}
    D --> P["Problem statement with metric"]
    P --> C["Candidate causes must explain every row"]
    C --> W["Work starts: human or agent"]

Worked example

Support reports "file uploads are broken". The request from a product manager is "increase the upload size limit, users are hitting it."

The engineer pulls the last 24 hours of upload attempts: 412,000 attempts, 9,100 failures (2.2%). Slicing:

DimensionIS (failure rate)IS NOT
App versioniOS 5.2: 11%iOS 5.1: 0.3%, Android: 0.4%, web: 0.2%
Networkcellular: 14% on 5.2Wi-Fi: 1.1% on 5.2
File sizeover 8 MB: 38% on 5.2 cellularunder 8 MB: 0.5%
Timesince 5.2 rollout reached 50% on Mondaybefore Monday
Server errorsnone; requests end with client disconnectnot 413 Payload Too Large

The size-limit theory fails the table: no 413s, and Android uploads the same sizes fine. The candidate that explains every row: 5.2 changed the upload client and the request timeout is now 30 seconds. An 8 MB file over a 2 Mbit/s cellular uplink needs at least 32 seconds, so larger files on slow links time out on the client.

Problem statement: "Since iOS 5.2, uploads over about 8 MB on cellular fail at 38% (baseline under 1%) because the client aborts after 30 s. Done when the 5.2.1 failure rate for that slice is under 1% over 48 hours. Non-goal: changing the server size limit." The fix is a timeout based on progress rather than total time, plus resumable uploads later. Raising the size limit would have done nothing.

In the wild

Trade-offs

ApproachStrengthsWeaknessesChoose when
Act on the request as givenFast, keeps stakeholders happy short termOften treats a symptom; wasted workThe request is cheap and reversible
Full written frameShared target, prevents reworkTakes time; can feel bureaucraticAnything costing more than a day or touching users
IS / IS NOT tableRapidly eliminates wrong causesNeeds telemetry with the right dimensionsPartial failures, "only some users" problems

Pitfalls in production

Leadership lens

Check yourself

  1. Why must a candidate cause explain the IS NOT entries, not just the IS entries?

    Show answer

    Many causes would produce the observed failure; only the true cause also explains why comparable places are unaffected, so IS NOT entries eliminate most wrong candidates.

  2. A server dashboard shows healthy p99 latency but users report timeouts. What framing question comes first?

    Show answer

    Where is the latency measured? Time spent before the application sees the request, such as SYN retransmits after accept queue overflow or network delays, never appears in server histograms.

  3. What turns "checkout is slow" into a usable problem statement?

    Show answer

    A metric with a baseline and a target, a scope (who and where), a start time, an impact estimate and a measurable "done" condition.

  4. Why ask an agent to restate the problem before it writes code?

    Show answer

    The restatement exposes misread requirements and hidden assumptions while they are cheap to correct, before they're spread across a large diff.

Sources and further study

Primary

Systems docs

Books

Other curricula

Watch

Free resources

Open in the interactive app: practice, quiz, mock interview →