Mapping requirements to building blocks
Open in the interactive app: practice, quiz, mock interview →In one minute
- In an interview, a concept is the answer to a requirement you heard or a number you computed. Every box should trace to a sentence like "reads outnumber writes 50 to 1 and may be 5 seconds stale, so I cache".
- A handful of signals triggers almost every concept: read-heavy with tolerable staleness, more writes or data than one node holds, slow work, one event reaching many users, a skewed key, a "never twice" rule, and more than one region.
- Underneath, most concepts are four primitives: copy, split, defer and agree. Copies go stale, splits break cross-partition operations, deferred work arrives late or twice, and agreement costs round trips.
- You have time for two or three proper deep dives. Rank the triggered concepts, go deep on the top ones, and name the rest in one sentence each.
- Every concept creates an obligation (a cache obliges invalidation, a queue obliges idempotent consumers). Budget for it or leave the concept out.
How it works
From signal to concept
Interviewers rarely say "use sharding". They say "200 million users, each posts a few times a day", and wait to see whether you notice the arithmetic has outgrown one machine. This table maps those signals to concepts and to the lessons that cover each in depth.
| Signal you hear or compute | Concept it triggers | Go deeper |
|---|---|---|
| Reads at least 10x writes, staleness of seconds is fine | Caching, read replicas, CDN | Caching, Scaling Read Traffic |
| Peak writes beyond about 10k per second, or data beyond a few TB | Partitioning and sharding, LSM storage | Scaling Write Traffic, Database Design |
| One key gets a large share of traffic | Hot-key splitting, request coalescing | Handling Hot Keys |
| Traffic jumps 10x in seconds (sales, launches) | Queues as buffers, admission control | Absorbing Traffic Spikes |
| Work takes longer than a request should wait | Queue plus workers, job state | Handling Long-Running Tasks, Scheduling Delayed and Recurring Jobs |
| One write must reach many readers | Fan-out on write or read | Fanning Out Updates |
| Others' changes visible within a second | Push (WebSocket, SSE) | Pushing Real-time Updates, Networking |
| "Never charge or sell twice" | Idempotency, conditional writes | Preventing Duplicate Processing, Preventing Double Booking |
| One action spans several services or stores | Sagas, outbox, change data capture | Coordinating Transactions Across Services, Keeping Data in Sync |
| Must survive losing a region | Multi-region replication and routing | Running Across Multiple Regions, Surviving Component Failures |
| Large blobs | Object storage, presigned URLs | Uploading and Serving Large Files, Streaming Video and Audio |
| Find by text, prefix or location | Inverted, trie or geo index | Search and Typeahead, Finding and Tracking Locations |
| Count, rank or ID huge streams | Sharded counters, sketches, ID schemes | Counting at Scale, Data Structures, Generating Unique IDs |
The five Concept Deep Dives sit underneath: Networking (what each arrow costs), Caching (the first tool for read load), API Design (requirements into an interface and access patterns), Database Design (access patterns into keys and partitions) and Distributed Systems (what happens when pieces disagree or fail). 30 Must-Know Concepts organises the same ideas by family; this chapter organises them by the signal that calls for them.
Rank before you explain
A 45-minute interview leaves about 10 to 20 minutes after the high-level design; one widely used framework budgets about 10 minutes for deep dives after roughly 20 to 30 minutes of requirements, API and high-level design (Hello Interview). A concept explained well, with mechanism, one number and failure mode, takes 4 to 7 minutes: two or three deep dives, not seven. Candidates who fail on depth rarely lack knowledge; they spend it on the wrong concept.
Rank the triggered concepts with three questions:
- What breaks the product if this is wrong? Double charges and lost messages outrank a slow profile page.
- What did the numbers force? At 40k writes per second, partitioning must appear and be justified.
- What is unusual about this problem? Caching shows up everywhere. The celebrity fan-out in a feed, the seat hold in ticketing or the merge in a collaborative editor is what the interviewer most wants to see.
Everything below the cut gets one sentence: "Static assets go through a CDN; happy to go deeper."
Concepts come with obligations
Each concept creates a question a strong interviewer will ask. A cache obliges invalidation and stampede handling, because cache-aside by itself guarantees no consistency with the database (Azure Architecture Center). Sharding obliges a key and a plan for cross-shard queries. A queue obliges duplicate handling, ordering and a dead-letter path; even a managed queue such as SQS standard only promises at-least-once delivery and tells you to make consumers idempotent (AWS docs). Replication obliges a statement of what a reader sees right after a write. Multi-region obliges conflict resolution. Budget the minute each obligation takes, or leave the box out.
Common mistakes
- Adding concepts that no requirement asked for, such as Kafka in a design with 50 writes per second.
- Explaining the textbook version of a concept instead of the version this problem needs ("caching is storing data in memory" rather than "cache-aside on the product page, 60-second TTL, delete on price change").
- Forgetting the obligation, so the interviewer has to ask "what if the consumer crashes after charging?"
Down the stack
Layer 1The contract
Each concept is a promise you can state as a testable sentence; that is what makes it an answer rather than a buzzword. Caching: "a read may be up to 30 seconds old; a write deletes the key". Sharding: "queries with the user ID touch one shard; others scatter to all 64". Queueing: "every accepted job runs at least once, usually within 10 seconds, possibly twice". Idempotency: "a repeat with the same key within 24 hours returns the first result and does nothing else". The window is a design choice you must state: Stripe may prune keys once they are 24 hours old, while a DynamoDB transaction's client token is honoured for only 10 minutes. Replication: "a committed write survives losing any one node; followers lag by up to a second". If you cannot finish the sentence, do not propose the concept.
Layer 2The mechanism
Almost every concept is built from four primitives. Copy: replicas, caches, CDNs, materialised views and search indexes store one fact in several places. Split: partitioning, sharding and consistent hashing divide data or work across machines. Defer: queues, logs, batching and change data capture move work out of the request path. Agree: transactions, consensus (Raft in etcd, ZAB in ZooKeeper), leases and compare-and-set make several actors accept one outcome. An etcd cluster needs a majority to accept any update, so three members survive one failure and five survive two (etcd FAQ).
The primitive predicts the obligation. Copies go stale: you need invalidation or a staleness budget. Facebook's memcache fleet, one of the largest copies ever run, still needed leases to stop stale sets and herds (NSDI 2013). Splits break cross-partition operations: you need a key that matches the dominant query. Deferred work arrives late or twice: you need idempotent consumers and lag monitoring. Agreement costs round trips and needs a majority: keep the agreed state small. Consensus keeps working while any majority is up and stops, without losing correctness, when it is not. An unfamiliar concept such as a CRDT becomes easy to place: it is copy without agree, hence its merge rules.
Layer 3The machine
The thresholds come from hardware. A DRAM lookup costs about 100 ns and a round trip inside one datacenter about 0.5 ms (latency table), so a database query over the network costs 0.5 to a few ms, so a cache hit is worth one to two orders of magnitude. A relational primary with its WAL on NVMe sustains thousands to low tens of thousands of small transactional writes per second, bounded by fsync latency, lock contention and replication: hence the 10k trigger. A server holds tens of TB of SSD, but backups, index rebuilds and failover of a multi-TB database get painful long before the disk fills. A 10 Gbit NIC carries about 1.25 GB/s, so a hot key returning 10 KB values saturates one cache node at around 100k reads per second, before its CPU runs out. A cross-region round trip of 60 to 150 ms (California to the Netherlands and back is about 150 ms in the same table) is why synchronous agreement across regions is reserved for small, critical writes.
Why it's built this way
One machine has finite memory, disk, network and failure tolerance, so at some scale you copy or split. The pieces then sit on a network that is slow and sometimes broken, so keeping them in step costs time. Most user-facing work tolerates a little delay or staleness, so deferring and copying buy capacity cheaply. A few operations (money, inventory, identity) cannot, so agreement is spent only there. That uneven need for exactness is why designs mix concepts per operation, and why ranking them matters.
Diagram
flowchart LR
R["Requirement or estimate"] --> S["Signal"]
S --> P{"Primitive"}
P -->|copy| C["Cache, replica, CDN"]
P -->|split| H["Partition, shard"]
P -->|defer| Q["Queue, log, fan-out"]
P -->|agree| A["Transaction, consensus, CAS"]
C --> O1["Obligation: staleness, invalidation"]
H --> O2["Obligation: key choice, cross-shard"]
Q --> O3["Obligation: duplicates, lag"]
A --> O4["Obligation: latency, availability"]Worked example
The prompt: ticket sales for a concert platform. You learn there are 20 million DAU browsing about 50 pages each per day, and big on-sales put 2 million people on one event page in the same minute for 50,000 seats. No seat may be sold twice and no payment taken twice.
- Reads: 20M × 50 / 86,400 is about 11,600 per second, read-heavy, with 10 seconds of staleness acceptable. That triggers caching and a CDN.
- On-sale: 2 million users refreshing one event every few seconds is hundreds of thousands of reads per second on one key. That triggers hot-key handling and spike absorption, typically a waiting room that admits users at a fixed rate.
- Seats: contention on 50,000 scarce items triggers double-booking prevention with a conditional write or an expiring hold.
- Payments: mobile retries trigger idempotency keys.
- At most 50,000 purchases per sale is a small write load, so sharding is not triggered. Say so; noticing what you do not need is a signal too.
With 18 minutes: the seat hold (correctness, unique, 7 minutes), the waiting room (forced by the numbers, 6 minutes), payment idempotency (4 minutes). Caching and the CDN get a sentence each. Each deep dive discharges its obligation: a hold expiring mid-payment, a refreshing user keeping their place, how long idempotency keys live.
In the wild
- Figma, 2024: splitting only when the numbers forced it. Figma's database stack grew almost 100x from 2020. The team first split groups of tables into their own databases, the cheap lever. Only when single tables reached terabytes and billions of rows, with vacuum trouble and writes nearing the RDS IOPS ceiling, did they shard horizontally. "Colos" of tables share one shard key so joins and transactions within a key still work, and a Go proxy routes queries. The first sharded table took about 9 months and went live in September 2023 (Figma).
- Slack, 2022: a copy and a split with different keys. Group DM membership was cached by channel ID, but the database was sharded by user. When a Consul rollout emptied part of the memcached fleet, every miss became a query across all shards, the database could not refill the cache, and many users could not start Slack. Each concept was fine alone; the obligation that failed was making the copy's key match the split's key (Slack Engineering).
- Twilio, 2013: deferred work without an idempotency guard. A Redis billing master came back read-only with balances at zero. Usage then triggered automatic card recharges that could not record the new balance, so recharges were attempted again and again; 1.4% of customers were affected. Billing was shut off, charges were refunded, and Twilio added fail-safes that block charges when a balance cannot be read or written (Twilio).
Trade-offs
| Strategy | Strengths | Weaknesses | Choose when |
|---|---|---|---|
| Deep on two or three concepts, one line on the rest | Shows real depth, fits the clock | Risk of picking the wrong two | Default; rank by damage, numbers and uniqueness |
| Broad tour of many concepts | Shows vocabulary | Shallow, invites "how exactly?" probes you have no time for | Rarely; only for a quick early sketch |
| Start from the simplest design and add concepts only when a number forces them | Every concept is justified | Can look slow if you over-explain the simple version | When you are unsure which concepts matter |
Pitfalls in production
- Résumé-driven architecture. Event sourcing or a service mesh gets adopted because it is interesting. Require a written trigger (a number or a rule) for each new primitive in design review.
- Copies with no staleness owner. A cache or search index ships with no agreed staleness, so "old data" tickets have no correct answer. Put the budget in the service contract.
- Splitting on the wrong key. A shard key chosen for writes makes the main read scatter to every shard. Choose it from the dominant query and test it against the second.
- Deferral without duplicate handling. Consumers retry and side effects happen twice. Make consumers idempotent before enabling retries.
- Agreement creeping into the hot path. A lock added for one rare case ends up on every request. Scope agreement to the operations that need it.
Leadership lens
- In design review, ask of each box: which requirement put it here, which primitive is it, and what obligation did it create?
- Prefer concepts the team can operate. A queue the team understands beats a stream processor that needs a specialist on call.
- Put the thresholds on dashboards ("we shard at 10k writes per second", alert at 60 percent) so the next concept arrives on a schedule, not in an incident.
- Teach new engineers the signals and the four primitives first. Technologies change; the signals do not.
Check yourself
Your estimate shows 300 writes per second and 2 TB of data after five years. Does that trigger sharding?
Show answer
Not by itself. Both numbers fit one well-provisioned primary with replicas; say so, and name the threshold at which you would partition and the key you would use.
Why does adding a cache oblige you to discuss consistency?
Show answer
A cache is a copy, and copies go stale. You must state how long a stale read may last and how writes invalidate or update the cached value.
Your deep-dive budget allows three topics, and the problem triggers six concepts. How do you pick?
Show answer
Rank by the damage of getting it wrong, then by what the numbers forced, then by what is unique to this problem. Go deep on the top three and name the rest in a sentence each.
Sources and further study
Primary
- Raft consensus algorithm: the paper, visualisation and the majority rule behind the "agree" primitive.
- Elhemali et al., Amazon DynamoDB (USENIX ATC 2022): copy and split at scale, with predictable latency as the contract.
Systems docs
- Stripe: Idempotent requests: a production idempotency contract, including how long keys live.
- Amazon SQS: At-least-once delivery: the duplicate obligation that comes with deferring work.
- etcd FAQ: quorum sizes and why clusters stay small and odd.
Books
- Designing Data-Intensive Applications (1st ed.), Martin Kleppmann, chapters 5 "Replication", 6 "Partitioning", 7 "Transactions" and 11 "Stream Processing": copy, split, agree and defer in depth.
- System Design Interview, Volume 1, Alex Xu, chapter 3 "A Framework For System Design Interviews": where concept selection sits in the hour.
Other curricula
- Hello Interview: Delivery framework: budgets each interview phase, leaving about 10 minutes for deep dives.
- roadmap.sh: System design: lists the same building blocks (caches, databases, queues, load balancers) as a map to study, without tying them to signals.
Watch
- System Design Interview: A Step-By-Step Guide (ByteByteGo): where concept selection fits inside the interview.
- Scaling Instagram Infrastructure (InfoQ): real signals (read load, fan-out, multi-region) and the concepts Instagram applied to each.
- CS75 Lecture 9: Scalability (Harvard, David Malan): copy, split and cache from first principles, step by step.
Free resources
- Hello Interview: Core Concepts: a concise interview-oriented tour of the same concepts.
- The System Design Primer: broad reference with links to deeper material for each concept.
- Designs, Lessons and Advice from Building Large Distributed Systems (Jeff Dean): the latency numbers and design patterns behind the thresholds in this chapter.
- On Designing and Deploying Internet-Scale Services (James Hamilton): the operational obligations each concept creates, from someone who ran them.