Engineering Atlas › System Design Fundamentals

Two-way messaging over WebSockets

14 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

How it works

The upgrade handshake

The client opens a TCP (and usually TLS) connection and sends an ordinary HTTP/1.1 GET:

GET /live HTTP/1.1
Host: chat.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: q4xkcO32u266gldTuKaSOw==
Sec-WebSocket-Version: 13
Origin: https://app.example.com

If the server agrees, it answers 101 Switching Protocols with Sec-WebSocket-Accept, which is the base64 SHA-1 of the client's key concatenated with a fixed GUID from the RFC (RFC 6455). That value proves the server really speaks WebSocket rather than being a confused HTTP cache echoing headers. From the next byte onward, the connection carries WebSocket frames, not HTTP.

Because it starts as HTTP, the handshake carries cookies, passes through port 443, and can be routed by path at a reverse proxy. Because it then stops being HTTP, everything after the handshake is invisible to HTTP-level tooling: no status codes per message, no access log line per message, no caching.

Messages and frames

Applications see messages; the wire carries frames. A message is one frame, or several frames where the first has a text or binary opcode, the rest are continuation frames, and the last has the FIN bit set. Fragmentation lets a sender stream a large message without knowing its length up front. Control frames (close, ping, pong) may be interleaved between fragments, so a 50 MB upload does not block a heartbeat.

Ping and pong are the liveness mechanism: either side may send a ping and the other must reply with a pong carrying the same payload. The close handshake is a close frame with a two-byte status code (1000 normal, 1001 going away, 1008 policy violation, 1009 message too big, 1011 server error), answered by a close frame, after which the server closes TCP. IANA has since registered 1012 "Service Restart" and 1013 "Try Again Later", which tell a client the server is going away on purpose and that it should back off before reconnecting (IANA registry). Code 1006 is never sent; browsers report it when the connection died without a close frame, which is what most real disconnects look like.

What you build on top

The protocol is deliberately thin. A production system adds a message envelope (type, id, sequence number), application-level acknowledgements where loss matters, a reconnect loop with jittered backoff, a resume protocol ("I last saw sequence 4,812 on channel 9"), authentication refresh for connections that outlive their token, and a routing layer that knows which server holds which user.

That routing layer is the fan-out problem shape again. When Alice posts to a channel with 3,000 members spread over 200 gateway servers, something must find those 200 servers and push to each. A phone network has the same shape: the switch that holds your call is not the one where the caller dialled, and a registry maps subscriber to switch. In WebSocket systems the registry is usually a pub/sub layer (Redis, NATS, Kafka) keyed by channel or user. Slack's version has stateful gateway servers holding the sockets and channel servers placed on a consistent-hash ring; each channel server host owns about 16 million channels at peak, and a message reaches clients worldwide in about 500 ms (Slack Engineering).

Down the stack

Layer 1The contract

On one connection, messages arrive complete, in order and uncorrupted, or the connection fails. That is TCP's guarantee lifted to message boundaries. Nothing survives a disconnect: messages the sender wrote but the network did not deliver are gone, and the sender cannot tell which. So the end-to-end contract you can promise users is at-most-once, unless you add sequence numbers and acks to get at-least-once with deduplication. Text messages must be valid UTF-8, and receivers must fail the connection if they are not.

Layer 2The mechanism

RFC 6455 defines each frame as: one byte holding FIN, three reserved bits (RSV1 to RSV3) and a 4-bit opcode (0 continuation, 1 text, 2 binary, 8 close, 9 ping, 10 pong); one byte holding the MASK bit and a 7-bit length; then an extended length (16 bits if the 7-bit value is 126, 64 bits if it is 127); then a 4-byte masking key if MASK is set; then the payload.

Every frame from client to server must be masked: each payload byte is XORed with key[i % 4], using a fresh random key per frame. Server frames must not be masked. Masking is not encryption; it exists because researchers showed, while the protocol was being designed, that attacker-controlled bytes on an upgraded connection could trick transparent proxies into caching forged HTTP responses (RFC 6455, section 10.3). Random masking makes client bytes on the wire unpredictable to the page's script.

Control frames carry at most 125 bytes and cannot be fragmented, so they always fit in a small fixed buffer and can jump between data fragments. Reserved bits must be zero unless an extension claims them; permessage-deflate (RFC 7692) uses RSV1 to mark a compressed message, negotiated in Sec-WebSocket-Extensions. RFC 8441 carries WebSockets over HTTP/2 as one stream opened with an extended CONNECT, so many sockets can share one TCP connection (RFC 8441). RFC 9220 (2022) applies the same extended CONNECT to HTTP/3, so a WebSocket can now ride a QUIC stream (RFC 9220).

Layer 3The machine

A small server-to-client message of 20 bytes costs 2 bytes of frame header, 5 bytes of TLS record header plus a 16-byte AEAD tag, and 40 to 52 bytes of TCP/IP header, so about 85 bytes on the wire. A client message adds 4 bytes of mask key. Compare a minimal HTTP request with cookies at 500 to 1,500 bytes, and you see why chatty clients move to WebSockets.

Unmasking is a byte-wise XOR, which servers vectorize to 16 or 32 bytes per instruction; at memory bandwidth it costs well under a microsecond per kilobyte. The real costs are per connection. An idle connection holds a file descriptor watched by epoll, a few KB of kernel socket state, the TLS session (around 30 to 50 KB in some libraries unless buffers are released when idle), and the application's read buffer and per-connection object. Gateways tuned for idle connections land between 10 and 60 KB each, so one million connections need tens of GB of RAM. File descriptor limits (ulimit -n, fs.file-max) and, at proxies, ephemeral ports (about 28,000 to 64,000 per source IP to one backend address) become real ceilings.

Slow readers are the classic failure. If a client on a weak mobile link stops reading, the kernel send buffer fills, write returns EAGAIN, and the application's outbound queue grows. Without a cap per connection, one bad network region can take a gateway's memory down with it. Browsers expose the client side of this as WebSocket.bufferedAmount, and the classic WebSocket API has no backpressure at all on receive: messages are pushed to onmessage as fast as they arrive. WebSocketStream adds stream backpressure, but MDN still lists it as non-standard with limited browser support (MDN).

Why it's built this way

  1. The web's infrastructure only reliably passed HTTP on ports 80 and 443, so the protocol had to start as HTTP and upgrade in place.
  2. Middleboxes that misread the upgraded bytes as HTTP were a security risk, so the handshake proves the server understands WebSocket and the client masks its frames.
  3. Interactive traffic is many small messages, so the frame header is 2 bytes for anything up to 125 bytes and only grows for large payloads.
  4. Long messages must not starve heartbeats or close requests, so data can be fragmented and small control frames can interleave.
  5. Applications differ wildly in what "reliable" means, so the protocol stops at ordered messages and leaves acks, resumption and routing to the application.

Diagram

sequenceDiagram
    participant C as Client
    participant G as "Gateway server"
    participant P as "Pub/sub"
    C->>G: GET /live, Upgrade websocket, Sec-WebSocket-Key
    G-->>C: 101 Switching Protocols, Sec-WebSocket-Accept
    G->>P: subscribe user and channel topics
    C->>G: text frame (masked) "send msg, seq 17"
    G->>P: publish to channel
    P-->>G: message for channel
    G-->>C: text frame (unmasked) "msg, seq 4813"
    G-->>C: ping
    C->>G: pong
    C->>G: close 1000
    G-->>C: close 1000, then TCP FIN

Worked example

A team chat product has 3 million concurrently connected clients at peak. Each client sends about one message a minute and receives about 20 (messages, typing indicators, presence).

Connections: at 40 KB per connection with TLS, 3 million connections is about 120 GB of RAM. With 64 GB gateways planned at 50 percent memory for connections, that is about 4 gateways' worth of memory but, with CPU for TLS and fan-out, a fleet of around 40 gateways at 75,000 connections each is a comfortable plan that survives losing a zone.

Traffic: 3 million x 20 / 60 is 1 million messages per second outbound at about 200 bytes each, 200 MB/s or 1.6 Gbit/s across the fleet. Inbound is 50,000 messages per second, which go through ordinary validation and persistence before fan-out.

Deploys: restarting one gateway drops 75,000 clients. If they all reconnect within one second, the remaining gateways absorb 75,000 TLS handshakes plus 75,000 resume requests at once. The client reconnect rule fixes that:

def reconnect_delay_ms(attempt, rand):
    base = min(30_000, 500 * 2 ** attempt)   # cap at 30 s
    return rand() * base                      # full jitter

The gateway also sends close code 1012 (or 1001 for older clients) and drains over several minutes instead of dropping everyone at once, and the resume request carries the last sequence number per channel so the server returns only the gap.

In the wild

Trade-offs

OptionStrengthsWeaknessesChoose when
WebSocketsFull duplex, tiny frames, binary support, low latency both waysStateful, no built-in resume, harder to load balance and deployChat, games, collaborative editing, live trading
SSEPlain HTTP, built-in reconnect and Last-Event-IDOne-way, text onlyFeeds, notifications, LLM streaming
Long pollingWorks through anything, stateless serversRequest per batch, higher latency and overheadFallback, low message rates
WebTransport (HTTP/3)Multiple streams and unreliable datagrams, no head-of-line blockingIn all major browsers only since March 2026 (MDN); thinner proxy and library supportMedia or games needing unreliable delivery

Pitfalls in production

Leadership lens

Check yourself

  1. Why must browsers mask client frames but servers not mask theirs?

    Show answer

    Masking stops a malicious page from choosing the exact bytes on the wire, which could fool intermediaries into treating them as HTTP and poisoning caches. The page's script controls client payloads, not server ones, so only client frames need it.

  2. A user's laptop sleeps for an hour and wakes up. The client shows "connected", but no messages arrive. What happened and how do you detect it?

    Show answer

    The TCP connection was dropped by a NAT or balancer without either end being told. Application pings with a pong deadline detect the dead connection, and the client then reconnects and resumes from its last sequence number.

  3. Why can a 125-byte limit on control frames matter for latency during large uploads?

    Show answer

    Because control frames are small and unfragmented, they can be interleaved between fragments of a large data message, so a ping or close never waits behind megabytes of data.

  4. Your chat service uses least-connections balancing. After a deploy, the newest gateway is overloaded and the old ones are idle. Why?

    Show answer

    Reconnecting clients all pick the gateway with the fewest connections, which is the freshly started one, so it gets the whole storm. Use slow start, handshake rate limits per node or random choice of two to spread the load.

Sources and further study

Primary

Systems docs

Books

Other curricula

Watch

Free resources

Open in the interactive app: practice, quiz, mock interview →