Engineering Atlas › System Design Interviews

Design live comments for streams

21 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

Requirements

Functional

Non-functional

Out of scope: the video pipeline itself (see Design YouTube and Media Streaming), reactions and like counters (see Distributed Counting), threaded replies, and the content classifiers behind moderation.

Back-of-envelope

Round assumed numbers for a large social platform.

API

POST /v1/videos/{video_id}/comments
  { client_comment_id, text, video_offset_ms }
  -> 201 { comment_id, status: "published" | "held" }

GET  /v1/videos/{video_id}/comments/recent?limit=50
  -> { comments: [ { comment_id, author, text, video_offset_ms, server_ts } ], cursor }

GET  /v1/videos/{video_id}/comments/replay?from_ms=600000&to_ms=660000
  -> { comments: [...], next_from_ms }

DELETE /v1/videos/{video_id}/comments/{comment_id}     -> 204

GET  /v1/videos/{video_id}/live         (Server-Sent Events)
  event: batch
  id: 1832
  data: { tick: 1832, comments: [...], dropped: 412 }

  event: delete
  data: { comment_id }

client_comment_id makes a retried post idempotent. video_offset_ms is where the poster's playhead was, which is what replay keys on. The live stream uses SSE because traffic is almost all server to client; posting goes over normal HTTP, which reuses the platform's auth, rate limiting and abuse checks. Serve it over HTTP/2: over HTTP/1.1, browsers allow only 6 connections per domain, so a viewer with several tabs open starves their own page loads, while HTTP/2 multiplexes the streams on one connection (MDN). The dropped field lets the client show "and 400 more" without receiving them.

Data model

comments lives in a wide-column store (Cassandra or similar). Partition key (video_id, minute_bucket), clustering key (video_offset_ms, comment_id). Columns: author_id, text, server_ts, status (published, held, deleted). Partitioning by video alone would put a hot stream's 10k writes/s on one replica set and make the partition unbounded; bucketing by minute of video time caps each partition and spreads a long stream over many. Replay reads one or two buckets at a time in offset order, which is a sequential scan.

comment_dedupe holds (author_id, client_comment_id) -> comment_id with a 24-hour TTL. A retry after a timeout returns the original id.

recent_comments is an in-memory list per video (Redis LPUSH plus LTRIM to 200, the capped-list idiom the Redis docs recommend (Redis docs)). Joining viewers read it instead of hitting the comment store, which turns a "join storm" at stream start into cache reads.

subscriptions exist only in memory. Each gateway keeps video_id -> set of connections. Each dispatcher keeps video_id -> {gateway_id: refcount} for the videos it owns. They are rebuilt from gateways' re-subscribe calls after a restart, so they never need a database.

comment_id is a time-ordered 64-bit id (Snowflake-style, see Generating Unique IDs), so ids sort by creation time within a video.

High-level design

flowchart LR
  V["Viewer app"] -->|"POST comment"| CS["Comment service"]
  CS --> DB[("Comment store by video + minute")]
  CS --> RC[("Recent list cache")]
  CS --> BUS["Pub/sub partitioned by video_id"]
  BUS --> D["Dispatcher (owns video_id)"]
  D -->|"one batch per tick"| G1["Gateway 1"]
  D --> G2["Gateway 2"]
  G1 -->|"SSE frame"| W1["Viewers"]
  G2 --> W2["Viewers"]
  W1 -.->|"join: subscribe"| G1
  G1 -.->|"first viewer of video"| D

Write path.

  1. The app shows the comment locally at once (optimistic echo), then posts it.
  2. The comment service authenticates, checks the per-user rate limit, deduplicates on client_comment_id, and runs a fast synchronous filter (block lists, link spam). Suspicious comments are stored as held and never published.
  3. It writes the comment to the store with quorum, pushes it onto the recent list, and publishes {video_id, comment} to the pub/sub topic. Then it returns 201.
  4. The dispatcher that owns this video_id (by consistent hashing) adds the comment to the video's pending buffer.

Read path.

  1. A viewer opens the live page: the app fetches the recent list, then opens the SSE stream to a gateway chosen by the load balancer.
  2. The gateway adds the connection to its local set for that video. If this is the gateway's first viewer of the video, it sends subscribe(video_id, gateway_id) to the owning dispatcher.
  3. Every tick (1 second), the dispatcher picks up to cap comments from the pending buffer, ranks them, serializes one batch, and sends it to every subscribed gateway.
  4. Each gateway writes the same frame bytes to every local connection watching that video, adding the viewer's own and friends' comments when it has them (see deep dive 2).
  5. When the last local viewer leaves, the gateway unsubscribes, and the dispatcher stops sending.

Deep dives

1. Routing comments to the right gateways

Option one is broadcast: every gateway receives every comment, 33M internal messages per second at 83k comments/s and 400 gateways, almost all thrown away. Option two is a pub/sub channel per video that gateways subscribe to directly (Redis pub/sub, for example). It works up to a point, but a broker with a million churning channels becomes the bottleneck, and gateways still receive every raw comment before any sampling.

I choose a dispatcher tier between the bus and the gateways. Slack's real-time stack has the same split: channel servers own channels by consistent hashing on the channel ID and forward each message to the gateway servers subscribed to that channel, which then deliver it to their own WebSocket clients (Slack Engineering). Videos are assigned to dispatchers by consistent hashing on video_id; the bus is partitioned the same way, so a dispatcher consumes only the partitions for the videos it owns. The dispatcher keeps a refcounted subscription table: gateway G is subscribed to video V while at least one viewer on G watches V. Refcounting matters because viewers join and leave constantly; the dispatcher should see one subscribe when G gets its first viewer of V and one unsubscribe when G loses its last, not a message per viewer. Discord applies the same idea inside its Erlang cluster: its Manifold library groups recipient processes by node and sends once per node, where local workers finish the delivery, after a single guild publish to 30,000 sessions had taken up to 2.1 seconds (Discord).

Where the subscription table lives is a real choice. I keep it in dispatcher memory and rebuild it from gateways after a failure. LinkedIn's live-video likes system keeps it in a shared key-value store instead, so any dispatcher can serve any video and a dispatcher crash loses nothing; it reports about 100,000 connections per frontend host, about 5,000 events per second per dispatcher, and about 75 ms p90 from dispatcher to client (LinkedIn, QCon London 2020). The shared store costs a lookup per publish; the in-memory table costs a rebuild storm after a crash.

For the top streams, sending 400 messages a tick is nothing. What can break is the inbound side, where 10k comments/s for one video land on one dispatcher; the fix is in Failure modes below.

2. Too many comments: sampling and ranking

At low volume, send everything. Above the cap, choose. Options: random sampling (fair, cheap, but shows spam and noise as often as good comments), first-come (biased to whoever has the lowest latency), or ranking. I rank by a score computed at write time: author reputation, whether the author is the streamer or a moderator, comment length and language match, and a spam probability. Within a tick, the top cap by score go out; ties go to the older comment so the feed reads roughly in time order. Comments older than a few seconds when the tick fires are dropped from the live view, since a comment about a goal arriving 15 seconds late is noise.

Personalization happens at the edge, because the dispatcher's batch is shared by millions. A viewer's own comment is echoed locally by the app. A friend's comment also travels over the platform's per-user notification channel to the friend's gateway, which merges it into that viewer's next frame; friend graphs are small next to stream audiences, so this is cheap.

Dropped comments are not lost. They are in the store and show up in replay and in the "all comments" view, which pages through the store instead of the live socket.

3. Replay in sync with the recording

During the live stream each comment is stamped with video_offset_ms from the poster's playhead. After the stream ends, the player asks for comments in windows of 60 seconds ahead of the playhead, using the (video_id, minute_bucket) partitions, and renders each comment when the playhead crosses its offset. Seeking discards the buffer and fetches the new window. Replay traffic is ordinary cacheable reads, so a CDN or read cache in front of the replay endpoint absorbs a recording that goes viral.

The subtle part is skew: viewers watch live with different delays, sometimes 20 seconds apart, so stamping the poster's playhead rather than server time makes replay line up with what the poster was reacting to. Deletes write a tombstone (status = deleted) instead of removing the row, which replay filters, and the live path broadcasts a delete event so clients remove the comment from screen.

Down the stack

Layer 1The contract

  • Post. A 201 means the comment is durable on a quorum of replicas and will appear in replay unless deleted. A retry with the same client_comment_id returns the same comment_id.
  • Live view. At most one batch per tick per viewer, at most cap comments per batch, and no comment older than max_age at selection time. Delivery is best-effort: a dropped frame is not resent.
  • Subscriptions. A gateway receives batches for video V exactly while it has at least one local viewer of V, give or take one tick.
  • Deletes. A deleted comment is never selected for a later batch, never returned by replay, and is removed from screens that already show it within one tick.

Layer 2The mechanism

SSE is plain HTTP: the response has Content-Type: text/event-stream and stays open, and each event is lines of event:, id: and data: ended by a blank line. The browser's EventSource reconnects automatically and sends the last id in a Last-Event-ID header; here the server uses it only to avoid re-sending the same tick, not to replay missed ticks. Two more parts of the spec matter at this scale. A retry: field sets the client's reconnection delay, so the server can hand each client a randomized delay before a deploy and spread the reconnects. And because some proxies drop idle HTTP connections, the spec suggests sending a comment line (starting with :) about every 15 seconds as a keepalive (WHATWG HTML Standard).

The bus can be Kafka with video_id as the partition key, so all comments for a video land on one partition in order and the owning dispatcher is a consumer of that partition (Kafka docs). The dispatcher's per-video state is a pending buffer, a subscription map with refcounts, and a min-heap of size cap for selection, which is O(n log cap) per tick. Gateways keep a hash map from video_id to a list of connection handles.

Layer 3The machine

  • Serialize once. For a video with 25,000 local viewers on one gateway, the gateway builds the frame bytes once, then calls write (or writev) on 25,000 sockets. Each call is roughly 1 to 3 microseconds of kernel time, so 25k writes cost 25 to 75 ms of CPU per tick, spread over cores.
  • TLS cannot be shared. Each connection has its own session keys, so the 2 KB frame is encrypted 25,000 times. AES-GCM with AES-NI runs at several GB/s per core, so 50 MB per tick is tens of milliseconds of one core. This, not JSON encoding, is the dominant per-viewer cost.
  • Bytes on the wire. A 2 KB frame is two TCP segments at a 1,448-byte MSS. On an idle connection the congestion window is large enough to send both at once, so a frame costs one round trip at most.
  • Idle cost. Each SSE connection holds kernel socket buffers (tunable down to a few KB when idle), TLS state, and the gateway's per-connection object: about 10 to 30 KB, so 250k connections fit in a few GB of RAM. epoll_wait lets a handful of threads wait on all of them.
  • Write path. The comment store is an LSM tree: a write is a commit-log append and a memtable insert, flushed later to immutable SSTables (Cassandra docs), so 83k writes/s across a cluster is routine.

Why it's built this way

  1. Humans read at most a few comments per second, so the live view is capped and ranked, and delivery cost follows viewers, not comments.
  2. Sockets are spread across hundreds of gateways, so routing goes to gateways with viewers, refcounted, instead of to individual viewers.
  3. TLS and syscalls cost per frame and per connection, so comments are batched into one frame per tick and serialized once.
  4. Sampling and lossy delivery mean the live feed cannot be the record, so every comment is stored durably, keyed for replay.
  5. Viewers sit at different points in the stream, so comments are keyed by video offset, not wall clock.

Failure modes

In the wild

Trade-offs

OptionStrengthsWeaknessesChoose when
SSE for live deliveryPlain HTTP, auto-reconnect, works through proxiesOne direction only; needs HTTP/2 to avoid the 6-connection limitComments and other server push
WebSocketTwo-way, binary framesOwn reconnect logic, harder through some proxiesInteractive features on the same socket
Long pollingWorks everywhereA request per update, high overhead at 1/sFallback only
Send every commentComplete, simpleCost scales with comments × viewersStreams under the cap
Ranked sample per tickBounded cost, better feedMost comments never seen liveLarge streams
Pub/sub channel per videoSimple, few moving partsBroker strain with millions of channelsSmall or medium scale
Dispatcher tier with refcountsGateway-level fan-out, scales to hot streamsAnother tier to run and rebalanceMillions of concurrent streams

Leadership lens

Check yourself

  1. Why is naive fan-out impossible for the top stream, and what bounds delivery cost instead?

    Show answer

    10k comments/s times 5M viewers is 5 x 10^10 deliveries per second, which is neither possible nor useful because nobody can read that fast. Capping each viewer at one frame per tick with a ranked sample of up to cap comments makes cost follow viewers, about 50M frames/s and roughly 100 GB/s of egress at peak across all gateways.

  2. Why put a dispatcher tier with refcounted subscriptions between the bus and the gateways instead of a pub/sub channel per video that gateways subscribe to directly?

    Show answer

    A broker with a million churning channels becomes the bottleneck, and gateways would still receive every raw comment before sampling. The dispatcher owns videos by consistent hashing, samples once per tick, and sends one batch per subscribed gateway, seeing only a subscribe on a gateway's first viewer and an unsubscribe on its last, at the cost of another tier to run and rebalance.

  3. The top stream's inbound comments overwhelm the one dispatcher that owns it. What breaks and how is it fixed?

    Show answer

    All of that video's comments, 10k per second, land on its owning dispatcher. Above a rate threshold, ingest for that video is split across N dispatchers that each keep a local top-K and merge candidates once per tick, the two-stage shape from Design Top K.

  4. Why is the comments table partitioned by (video_id, minute_bucket) and clustered by video_offset_ms rather than partitioned by video and ordered by server time?

    Show answer

    Partitioning by video alone would put a hot stream's 10k writes/s on one replica set and grow an unbounded partition; minute buckets cap each partition and spread a long stream. Keying by the poster's playhead offset lets replay line up with what the poster was reacting to, since live viewers can be 20 seconds apart, and replay reads one or two buckets sequentially.

Sources and further study

Primary

Systems docs

Books

Other curricula

Watch

Free resources

Open in the interactive app: practice, quiz, mock interview →