Engineering Atlas › System Design Interviews

Delivering live updates to clients

13 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

How it works

The four transports

Short polling: the client asks "anything new?" every N seconds. Trivial, but latency averages N/2, and 1 million clients polling every 5 seconds make 200,000 mostly empty requests per second.

Long polling: the server holds each request open until there is data or a timeout of perhaps 30 seconds, and the client immediately asks again. Latency drops to one round trip and it works through any HTTP proxy, but each delivery costs a full request, and messages arriving between requests must be buffered. The IETF wrote these costs down in 2011, including proxies that buffer or time out held responses (RFC 6202).

Server-Sent Events: one long-lived HTTP response with Content-Type: text/event-stream, onto which the server writes data: lines. It is one-way and text-only, and the browser's EventSource reconnects automatically, sending Last-Event-ID so the server can resume; a 204 No Content reply tells it to stop retrying (WHATWG HTML). Over HTTP/1.1 a browser allows only six such connections per domain across all tabs, so a seventh tab hangs; over HTTP/2 they are streams on one connection, and the stream limit is negotiated, typically 100 (MDN).

WebSockets: an HTTP/1.1 request with Upgrade: websocket switches the connection to a framed, bidirectional protocol, suited to chat, games and collaborative editing. That upgrade is not the only way in any more: since 2018 a WebSocket can also run as one stream of an HTTP/2 connection using an extended CONNECT with :protocol: websocket (RFC 8441), and the same trick was defined for HTTP/3 (RFC 9220), so check what your proxies actually speak. It has no built-in resume or reconnect, so you build acknowledgements, backoff and offset tracking yourself.

A fifth option is arriving: WebTransport runs over HTTP/3 and QUIC, offers several streams plus unreliable datagrams, survives a switch from Wi-Fi to cellular, and became Baseline across major browsers in March 2026 (MDN). It suits games and live media more than chat, and many corporate networks still block UDP, so keep a WebSocket fallback.

A sensible default: SSE for server-to-client feeds, WebSockets when the client sends often, long polling as a fallback, plain polling when 30 seconds of delay is fine. Backgrounded mobile apps keep none of these alive, so use APNs or FCM as a wake-up signal.

Connection gateways

A dedicated gateway tier terminates sockets. Each gateway holds hundreds of thousands of idle connections, authenticates once, and maps user or channel to connections. It does little else, so it can be deployed and drained independently of the services producing events. Slack is a public example: gateway servers hold WebSocket subscriptions and are the only tier spread across regions, while channel servers, placed by consistent hashing, own each channel's recent history and push to every subscribed gateway, for global delivery in about 500 ms (Slack Engineering).

Routing events to the right gateway

Bob's socket lives on one of perhaps 200 gateways. How does a message reach it?

Presence

Gateways heartbeat connected users into a presence store holding user -> last_seen with an expiry of, say, 60 seconds; online means a recent heartbeat. Presence changes are events fanned out only to contacts or room members, and throttled so a user flapping on a train does not broadcast 40 transitions a minute. Slack narrows it further: a client only receives presence for users currently visible on its screen (Slack Engineering).

Reconnect and resume

Every event gets a monotonically increasing offset per channel. The client stores the highest offset it processed and sends it on reconnect; the server replays newer events from a bounded buffer (Redis Streams, Kafka or an in-memory ring). If the offset is older than the buffer, the server says "resync": the client refetches current state through a normal API call and continues from the latest offset. Reconnects use exponential backoff with jitter to avoid stampedes.

How it shows up in interviews

Triggers: WhatsApp, Slack, live comments, driver tracking, Google Docs, auctions, notifications. A strong answer justifies the transport, separates gateways from business services, routes events to the right gateway, sizes connections, and resumes with offsets. Probes: a gateway dies, a user on three devices, deploying gateways, guaranteeing nothing is missed.

Down the stack

Layer 1The contract

The socket promises nothing durable: a push is at most once, and a TCP write that succeeded locally may never reach a phone in a tunnel. The real contract comes from the log: every event is stored with an offset, delivered at least once to connected subscribers in offset order per channel, and recoverable after a disconnect if the gap fits the replay window. Clients de-duplicate by offset. Latency targets are typically a few hundred milliseconds.

Layer 2The mechanism

The WebSocket handshake (RFC 6455) sends Sec-WebSocket-Key; the server answers 101 Switching Protocols with Sec-WebSocket-Accept, the base64 SHA-1 of the key plus a fixed GUID. Frames carry a FIN bit, an opcode (text, binary, close, ping, pong), a mask bit and a 7, 16 or 64-bit length; client frames are XOR-masked with a 4-byte key to protect intermediaries from cache poisoning. SSE is simpler: id:, event:, data: and retry: lines, a blank line ending each event, with id: becoming the next Last-Event-ID. A gateway typically runs an event loop per core and keeps channel -> set(connection) with a reference count per channel to drive subscribe and unsubscribe.

Layer 3The machine

Each connection is a file descriptor and a kernel socket with buffers autotuned from tcp_rmem and tcp_wmem; idle sockets hold a few kilobytes, so 500,000 connections cost a few GB of kernel and application memory. epoll_wait returns only ready descriptors, at cost proportional to ready sockets rather than total, which is why one process can hold so many. Limits bite elsewhere: ulimit -n, and load balancers or NAT gateways capping flows or source ports. Proxy idle timeouts silently kill quiet connections: an AWS Application Load Balancer defaults to 60 seconds, and HTTP/2 PING frames do not reset it (AWS), so heartbeat with real data every 20 to 30 seconds. The SSE spec suggests a comment line about every 15 seconds for the same reason. Managed gateways add hard caps: an API Gateway WebSocket closes after 10 idle minutes and after 2 hours no matter what, with 32 KB frames (AWS), so resume is a routine path, not an edge case. Heartbeats drain mobile batteries, another reason background delivery uses APNs or FCM. A mass reconnect is a TLS handshake storm costing real CPU.

Why it's built this way

HTTP cannot initiate from the server, so clients hold a connection open. Open connections are cheap in memory but tied to one machine, so a routing layer must find that machine. Machines and networks fail, so the socket cannot be the source of truth, and a durable log with offsets is. Logs cannot be kept forever, so the replay window is bounded, and clients outside it resync from current state. Reconnects arrive in bursts, so clients add jitter and servers drain gradually.

Diagram

sequenceDiagram
    participant C as Client
    participant G as Gateway
    participant B as Pub/sub bus
    participant S as Chat service
    participant L as Channel log
    C->>G: connect, resume room:42 after 117
    G->>L: read room:42 offsets > 117
    L-->>G: 118, 119
    G-->>C: events 118, 119
    G->>B: subscribe room:42
    S->>L: append message, offset 120
    S->>B: publish room:42 offset 120
    B->>G: offset 120
    G-->>C: event 120

Worked example

A food delivery app shows live courier location and order status. At dinner peak 2 million customers have the tracking screen open. Couriers send location every 4 seconds; 150,000 couriers are active.

Transport: customers only receive, so SSE over HTTP/2. Couriers send frequently, so their app uses a WebSocket.

Gateways: at a conservative 100,000 connections each, 2 million customers need 20 gateways, plus 30% spare: 26. At roughly 20 KB per connection including TLS state, that is about 2 GB per gateway.

Routing: each screen subscribes to order:{id}. Couriers send 37,500 updates per second, each published to one or two active orders, so about 50,000 publishes per second on a sharded NATS or Redis bus, usually reaching one gateway each.

Resume: each order channel keeps its last 50 events in a Redis Stream. A phone that switched networks resumes from offset 312 and gets 313 to 318; a customer back after an hour gets "resync" and reloads the order page.

Deploys: draining one gateway sends 100,000 clients to reconnect. With reconnect delay randomised between 0 and 10 seconds, the other gateways see about 10,000 new TLS handshakes per second in total, which they can absorb.

In the wild

Trade-offs

OptionStrengthsWeaknessesChoose when
Short pollingSimplest, cacheable, statelessLatency of N/2, wasted requestsUpdates every minute or so
Long pollingWorks everywhere HTTP worksRequest per message, bufferingFallback, restrictive networks
SSEBuilt-in reconnect and Last-Event-ID, plain HTTPServer to client only, textFeeds, notifications, dashboards
WebSocketsBidirectional, low overhead per messageCustom resume, harder load balancingChat, games, collaboration
Mobile push (APNs, FCM)Works when app is closedBest effort, rate limited, payload limitsBackground alerts and wake-ups

Pitfalls in production

Leadership lens

Check yourself

  1. Why does a real-time system need offsets if TCP already guarantees ordered delivery?

    Show answer

    TCP only orders bytes within one connection. When the connection drops and a new one opens, possibly to another gateway, offsets tell the server exactly what the client missed.

  2. When is SSE a better choice than WebSockets?

    Show answer

    When data flows mainly server to client: it is plain HTTP, works with existing proxies and HTTP/2 multiplexing, and has reconnect with Last-Event-ID built in.

  3. Why do gateways subscribe per channel instead of receiving every event?

    Show answer

    Broadcasting every event to every gateway multiplies bus traffic by the gateway count; per-channel subscription delivers each event only where someone is listening.

  4. What should the server do when a reconnecting client's offset is older than the replay buffer?

    Show answer

    Tell the client to resync: fetch current state through a normal read, then continue streaming from the latest offset.

Sources and further study

Primary

Systems docs

Books

Other curricula

Watch

Free resources

Open in the interactive app: practice, quiz, mock interview →