Engineering Atlas › System Design Interviews

Design a virtual waiting room

19 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

Requirements

Functional

Non-functional

Out of scope: the purchase flow itself (seat holds, payments; see the flash sale and double-booking lessons), identity verification of buyers, and resale.

Back-of-envelope

Assume a stadium on-sale: 60,000 tickets, 3 million people who want them, most arriving in the first 10 minutes.

API

POST /waitingroom/{event}/join
  headers: Cookie: wr_ticket=<token>        // optional; present on return visits
  body:    { "challenge": "<proof-of-work or CAPTCHA solution>" }
  200 -> { "state": "waiting", "ticket": "<token>", "poll_after_ms": 8000 }
  200 -> { "state": "admitted", "pass": "<token>", "expires_at": 1767225600 }
  429 -> { "state": "retry", "retry_after_ms": 3000 }    // join storm shed

GET /waitingroom/{event}/status
  headers: Cookie: wr_ticket=<token>
  200 -> { "state": "waiting", "position_range": [18200, 18800], "eta_min": 17,
           "sold_out_likelihood": "high", "poll_after_ms": 10000 }
  200 -> { "state": "admitted", "pass": "<token>", "expires_at": 1767225600 }
  401 -> { "state": "invalid" }       // bad signature
  410 -> { "state": "expired" }       // missed heartbeat window; must rejoin

Protected origin (every request):
  Cookie: wr_pass=<token>             // verified at the edge with the shared key
  no valid pass -> 302 to /waitingroom/{event}

Operator (authenticated)
PUT /admin/events/{event}  { "capacity": 8000, "max_admit_per_sec": 25,
                              "ordering": "bucketed_random", "bucket_seconds": 60,
                              "session_ttl_s": 600, "heartbeat_ttl_s": 45 }
POST /admin/events/{event}/pause

Position is returned as a range, not an exact integer. Exact numbers imply a precision the system does not have, and they invite people to compare notes and argue.

Data model

The system holds little durable data. Most of it is counters and signed blobs.

The partition key is the event id, then the bucket id. One on-sale is one hot partition, so counters are further split per edge location and summed periodically.

High-level design

flowchart LR
  U["Visitor browser"] --> E["Edge worker"]
  E -->|"valid pass"| O["Protected origin"]
  E -->|"no pass: mint ticket"| Q["Waiting page"]
  Q -->|"poll or SSE"| E
  E <--> C["Bucket counters"]
  A["Admission controller"] --> C
  M["Origin metrics"] --> A
  A -->|"open slots, rate"| C
  K["Config and signing keys"] --> E

Join path. The visitor hits the protected URL. The edge worker checks for a valid pass cookie. If there is one, the request goes through. If not, the worker checks whether the site is full. If the active count is below capacity and nobody is queued, it mints a pass and lets the visitor in. Otherwise it verifies a join challenge, mints a ticket for the current arrival bucket, increments that bucket's counter, and serves a static waiting page. Everything here is served from the edge, so the origin never sees the surge.

Admission path. A controller runs every second. It reads origin health and the live pass count, computes the number of free slots (bounded by max_admit_per_sec), and advances the admission watermark: the oldest bucket that is not yet fully admitted. Tickets at or behind the watermark are admitted. A ticket in the bucket currently being drained is admitted by a deterministic pseudo-random draw against the remaining fraction.

Read path (status). The waiting page polls status with its ticket. The edge verifies the signature, compares the ticket's bucket and draw with the watermark, and answers waiting with a range, or admitted with a fresh pass. The ticket cookie being refreshed is the heartbeat. No database row is read for a typical poll, only the cached watermark and counters.

Deep dives

1. Where the queue lives: signed tokens and counters

Option A: a central queue (a list or sorted set in Redis, one entry per visitor). It gives exact positions and trivial FIFO. But 3M entries and 300,000 polls per second hit one data structure, every poll is a rank query, and failover loses people's places.

Option B: stateless tokens. The ticket itself says when the holder arrived. The server only needs to know how far admission has progressed. Cloudflare's design is the model: each visitor gets a cookie with the timestamp of first arrival, and visitors are bucketed by arrival minute with the oldest buckets admitted first.

Choice: B with counters. Per-bucket counters give approximate positions at constant cost per poll. The price is that position is an estimate, and that the signing key becomes critical: leak it and anyone can mint a ticket for the front. Keys carry ids and validity windows so they can rotate while old tickets stay valid for a few minutes. Cloudflare also rejected an eventually consistent store for the global counts because stale reads over-admit. Each data center reports to one coordinating object per event, and the slice each location may admit follows its recent share of traffic.

2. Fairness and bots

Pure FIFO rewards the fastest connection and the best script. A bot that joins within a few milliseconds of the opening beats a person who clicked a second later. Cloudflare's docs describe FIFO as rewarding the people who queue first, and Random as the more equitable choice.

Bucketed random is the compromise: order by arrival bucket (so someone who arrived 40 minutes ago is not overtaken by someone who arrived a minute ago), and shuffle inside a bucket with a random draw fixed at join time. For the opening minute you can widen the first bucket so that "everybody within the first 5 minutes" is one lottery. This removes the millisecond race.

Bots then switch to quantity: many tickets per person. Defences stack, and none is perfect:

Choice: bucketed random with a challenge on join and per-identity de-duplication. State the residual: a determined, well-funded operator will still get through, and the aim is to raise their cost above the resale margin.

3. Admission rate tied to capacity, and abandonment

The room admits at the rate the origin can absorb. Three controllers, from simplest:

Choice: rate-limited static as the floor, with a feedback multiplier that can only reduce the rate automatically. Increases are a human decision, or a slow ramp.

Abandonment is the thing that makes capacity maths fail. A third of the queue may be gone within minutes. If their places stay reserved, admitted passes go unused and the origin idles while the line stalls. The fix is the heartbeat: each status poll refreshes a last-seen time, and a ticket that misses its window (45 s here) is dropped. The same applies after admission: an unused pass should expire in a minute or two, and a shopper's session should have a hard ceiling so a slot always comes back.

Down the stack

Layer 1The contract

  • A visitor with a valid pass is served by the origin. A visitor without one never reaches the origin, except through a documented bypass (health checks, payment callbacks).
  • A ticket confers a place, never access. A pass confers access for a bounded time, and only the room can mint one.
  • Admission is monotone: if ticket A is ahead of ticket B under the policy and both stay live, A is admitted no later than B (exactly for FIFO, in distribution for bucketed random).
  • The number of live passes never exceeds capacity, and passes minted in any second never exceed max_admit_per_sec plus a small stated overshoot from replication delay.
  • Joining twice returns the same ticket. Missing the heartbeat window loses the place, and the response says so (410).

Layer 2The mechanism

A ticket is base64(payload) "." base64(HMAC-SHA256(key_id, payload)). HMAC with a shared secret is enough because the edge both mints and verifies, so there is no need for public-key signatures; a JWT with HS256 (RFC 7519) is the same idea in a standard envelope. The payload holds the arrival bucket and a random draw taken at join, so ordering needs no server memory. Verification recomputes the MAC and compares in constant time, so response timing does not leak the signature.

Each admission tick does this: expire stale heartbeats and passes, add the elapsed time times the rate to a credit counter (capped at one second of burst), compute slots = min(capacity - active, floor(credit)), then advance the watermark by slots through the buckets in order. A pass is minted lazily, when the holder next polls and finds its ticket behind the watermark. This keeps the admission loop O(buckets), not O(people).

The status channel can be plain polling with a poll_after_ms hint, or server-sent events, which hold one cheap connection per visitor and push admitted the moment it happens. Joins shed under overload return 429 with a retry delay.

Layer 3The machine

At 20,000 joins per second the limit is accepting connections and TLS handshakes, not hashing. A TLS 1.3 handshake costs a few hundred microseconds of server CPU, while an HMAC-SHA256 over 100 bytes takes roughly 1 microsecond. Session resumption and HTTP/2 multiplexing keep the 300,000 polls per second on warm connections instead of a new TCP handshake each.

The waiting page is a few KB of static HTML and JS served from the edge cache, so 3M visitors cost cache hits, not origin work. A status response is a few hundred bytes, which fits in one TCP segment (MSS around 1,460 bytes) and one TLS record. Each SSE connection holds a socket and roughly 10 to 30 KB of kernel and TLS state, so one tuned edge node (file descriptors, epoll) holds hundreds of thousands.

Why it's built this way

  1. The surge is 100x the origin's capacity, so the origin cannot be the thing that sorts it out. The queue goes to the edge, where capacity is cheap and already global.
  2. 3M people polling needs lookups that cost nothing, so the token carries the state and the server keeps counters. That makes verification stateless and lets any edge answer any poll.
  3. Stateless tokens are forgeable if the key leaks and cannot be revoked one by one, so tokens are short-lived, keys are rotated by id, and passes are bound to a short expiry.
  4. Fairness is chosen by the business, and the cheapest attack is speed, so ordering uses buckets and randomness and joining has a cost.
  5. The origin's real limit moves, so the rate is a controlled dial with automatic decreases and human increases.

Failure modes

In the wild

Trade-offs

OptionStrengthsWeaknessesChoose when
FIFO orderingEasy to explain; rewards early arrivalRewards speed and bots; the opening second decides everythingLow bot risk, small events
Bucketed randomKills the millisecond race; early arrivals still favouredHarder to explain; positions are estimatesHigh-demand on-sales
Central queue (sorted set)Exact positions, simple codeSingle hot structure; polls are rank queries; failover loses placesSmall rooms under about 50,000 people
Stateless tokens + countersConstant cost per poll; any edge answersApproximate position; key management is criticalLarge, spiky events
SSE or WebSocket pushInstant admit; far fewer requestsMany open sockets; needs reconnect handlingWaiting pages that stay open for long
Feedback-controlled rateTracks real origin healthCan oscillate; harder to testAfter static is proven, with human-approved increases

Leadership lens

Check yourself

  1. Your origin handles 6,000 concurrent shoppers and the mean session is 5 minutes. What admission rate keeps it at capacity, and why does a longer real session mean the room must slow down?

    Show answer

    By Little's law, 6,000 / 300 s = 20 admissions per second. If sessions stretch to 10 minutes, the same rate would fill twice the capacity at steady state, so the rate must fall to 10 per second, or the active count will cross the limit.

  2. Why is pure FIFO a weak fairness policy for an on-sale, and what does bucketing change?

    Show answer

    FIFO is decided by arrival order at millisecond precision, which a script wins. Bucketing groups arrivals into windows such as one minute and shuffles inside each window, so the speed advantage within a window vanishes, while people who arrived earlier still go first.

  3. The counter store becomes unreachable during an event. Should edges fail open or closed, and what does the visitor see?

    Show answer

    Closed: freeze the watermark and admit nobody new, because opening the gate lets the whole surge hit the origin. Visitors keep seeing the waiting page with a status message, and existing pass holders continue.

  4. Why does the status poll dominate the load, and what two changes cut it?

    Show answer

    3M waiting visitors polling every 10 s is 300,000 requests/s versus a few thousand joins/s. Longer intervals far from the front, and server-sent events, cut it several times over.

Sources and further study

Primary

Systems docs

Books

Other curricula

Free resources

Open in the interactive app: practice, quiz, mock interview →