Engineering Atlas › System Design Interviews

Design a multiplayer online chess service

18 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

Requirements

Functional:

Non-functional:

Out of scope: puzzles, chat moderation and the analysis board.

Back-of-envelope

Games: assume 10 million games a day, about twice the 5 million a day that Lichess has cited publicly (Lichess at Big Techday 2022). That is 10,000,000 / 86,400 = about 116 new games per second, with an evening peak of perhaps 3x, so 350 per second.

Concurrency: if the average game lasts 10 minutes, the average number of live games is 116 x 600 s = about 70,000, and about 210,000 at peak. That is 420,000 players in games. Adding lobby users and spectators, plan for about 1 million concurrent WebSocket connections.

Moves: an average game has about 80 plies (40 moves each). 10 million x 80 = 800 million moves a day, about 9,300 per second on average and 28,000 at peak. Bullet games move faster, so plan for 50,000.

Storage: a move fits in 2 bytes (from-square, to-square and promotion piece fit in 16 bits) and a clock reading per ply in 2 more (tenths of a second, up to about 109 minutes). 80 plies x 4 bytes = 320 bytes, plus about 200 bytes of metadata, is about 0.5 KB per game. That is 5 GB per day, 1.8 TB a year and about 9 TB over 5 years.

Bandwidth: a move message is about 100 bytes. At 50,000 moves per second, each sent to two players and on average four spectators, that is 50,000 x 6 x 100 bytes = 30 MB/s. Bandwidth is trivial; the hard resource is open connections, at roughly 10 to 50 KB of kernel and application memory each, so 1 million connections need 10 to 50 GB of RAM across the socket tier.

API

POST /v1/seeks            { time_control: "3+2", rated: true, rating_range: null }
  -> 202 { seek_id }      (match arrives later over the socket)
DELETE /v1/seeks/{id}

WS /v1/socket             (one socket per client, multiplexed)
  client -> { t: "move", game: "g7Hq2", ply: 23, uci: "e7e8q", lag_ms: 41 }
  server -> { t: "move", game: "g7Hq2", ply: 23, uci: "e7e8q", san: "e8=Q+",
              clock: { white_ms: 151200, black_ms: 143900 } }
  server -> { t: "reject", game: "g7Hq2", ply: 23, reason: "illegal" }
  server -> { t: "end", game: "g7Hq2", result: "1-0", reason: "timeout" }
  client -> { t: "resign" | "draw_offer" | "draw_accept" | "takeback", game, ply }
  client -> { t: "ping", sent_at }   server -> { t: "pong", sent_at }

GET  /v1/games/{id}            -> { white, black, moves, clocks, result }
GET  /v1/games/{id}.pgn        -> PGN text
GET  /v1/users/{id}/games?cursor=...

UCI notation (e7e8q) is unambiguous and trivial to parse; SAN is for display. The ply number is the idempotency and ordering key, and ping and pong let the server estimate round-trip time.

Data model

games(game_id PK, white_id, black_id, time_control, rated, status, result, termination, moves BLOB, clocks BLOB, initial_fen, started_at, ended_at). Partition key game_id, a short random ID. Moves and clocks are packed byte arrays because a game is always read whole.

user_games(user_id, ended_at DESC, game_id, opponent_id, result): partitioned by user, clustered by time, written once at game end.

ratings(user_id, perf) with rating, deviation, volatility, games: one row per user per time-control family. Glicko-2 needs all three numbers, and the rating deviation is what makes a new player's rating move fast.

live_games lives in memory on the owning game server: the board (12 bitboards of 8 bytes, plus side to move, castling rights, en passant square and halfmove counter), a list of Zobrist hashes of past positions for repetition, both clocks, the ply, pending offers and the connected sockets. A durable copy of each accepted ply is appended to a replicated move log (a Redis stream with replicas, or a Kafka topic partitioned by game_id) so a crash loses nothing that was acknowledged.

High-level design

flowchart LR
  P["Players and spectators"] --> LB[L4 load balancer]
  LB --> WS[Socket servers]
  WS --> MM[Matchmaker]
  WS --> GR[Game router]
  GR --> GS["Game servers (one actor per game)"]
  MM --> GS
  GS --> ML[("Replicated move log")]
  GS --> PS["Pub/sub per game"]
  PS --> WS
  ML --> AR[Archiver]
  AR --> DB[("Games archive")]
  AR --> RT[Rating service]
  AR --> AC["Engine analysis (anti-cheat)"]
  RT --> RDB[("Ratings")]

Write path for a move: the client sends {move, ply, lag_ms} on its WebSocket. The socket server looks up the owning game server (consistent hash of game_id, with a lease so ownership is unique) and forwards the message. The game actor checks the ply, checks the turn, stops the mover's clock with lag compensation, validates legality with its move generator, applies the move, adds the increment, appends the ply to the move log, and publishes the new state on the game's pub/sub channel. Socket servers subscribed to that channel push it to both players and to spectators.

Read paths: a reconnecting player gets the full move list and clocks from the actor and replays them. Spectators of popular games go through a fan-out tier, sometimes deliberately delayed. History reads go to the archive.

When a game ends, the actor emits an end event. The archiver writes the packed game and the user_games rows, the rating service applies Glicko-2 to both players, and rated games are queued for engine analysis.

Deep dives

One owner per game and crash recovery

Option one is a stateless service that reads the game from a database, validates, and writes back with a version check. It survives crashes, but every move pays a database round trip, races need optimistic retries, and clock logic spreads across processes.

Option two, which I choose, is an actor per game in memory, owned by exactly one game server. All messages for a game go to its mailbox and are processed one at a time, so there are no locks and no races. Ownership comes from consistent hashing plus a lease in a coordination service (etcd or ZooKeeper). A server that loses its lease must stop accepting moves before the lease expires, which is why the lease is checked against a local monotonic clock with a safety margin.

Durability: the actor appends each accepted ply to the replicated move log (about 1 ms within a zone) before broadcasting it, so no client sees a move that could be lost. After a crash, the new owner replays the log, restores the logged clocks, and pauses both clocks during recovery. Clients reconnect and resend unacknowledged moves by ply; duplicates are dropped. If recovery cannot save a game fairly, abort it: Lichess aborts games in which someone lost on time because Lichess itself restarted (Lichess FAQ).

Clocks, lag and flagging

The server records when each turn starts. When a move arrives, elapsed time is now - turn_start. A player on a 150 ms round trip loses about 75 ms per move in transit, 3 seconds over a 40-move bullet game. So the server subtracts a lag estimate: the client reports how long the move spent in transit by its own measurement, and the server caps that credit at a value derived from the ping history (for example the player's recent median round-trip plus a margin, never above a fixed maximum such as 500 ms). The cap matters, because a client that claims huge lag would otherwise get free time. Lichess implements the cap as a per-player quota that behaves like a token bucket: each move earns credit (up to one second, less in faster time controls; 0.39 s per move in 1+0 bullet), compensation can never exceed the credit banked, and the bank holds at most seven moves' worth (scalachess LagTracker).

Flagging is decided only by the server. Each game actor registers its next possible flag time (turn_start + remaining) in a timing wheel; when it fires, the actor checks the clock and ends the game. A move arriving after the flag time is rejected even if the client still showed time. A flag is a draw, not a loss, if the opponent could not checkmate by any possible series of legal moves, for example with a bare king (FIDE Laws, article 6.9). Lichess notes this is hard to decide automatically in rare positions and by default sides with the player who did not run out of time (Lichess FAQ).

Matchmaking and ratings

A seek joins a pool keyed by (time_control, rated, variant). Instead of matching on arrival, the matcher runs a batch cycle about once a second per pool. It sorts seekers by rating and greedily pairs neighbours whose gap is within both players' current windows. Each window starts at about 100 points and grows with wait time, so a 2,700 player at 4 a.m. still gets a game within a minute. A peak pool holds a few thousand seekers, so one process per pool sorts them in under a millisecond.

Ratings use Glicko-2, which tracks rating, deviation and volatility (Glickman). Lichess starts players at 1500 with a deviation of 1000 and treats a rating as provisional while the deviation is above 110 (Lichess FAQ). Updates happen after the game in the rating service, as a conditional write per player, so they never slow the move path. Anti-cheat is also asynchronous: rated games are queued for engine analysis, and the move-by-move agreement with the engine plus timing patterns feed a model and human review. Lichess runs this at scale with fishnet, which distributes Stockfish analysis to volunteer machines that each request a personal key (fishnet).

Down the stack

Layer 1The contract

For each game, the server accepts at most one move per ply, in order, and every subscriber receives accepted moves in ply order with no gaps. A move is broadcast only after it is durable in the move log. The clock reported with each move is the server's value after lag compensation and increment. A resent move with an already-accepted ply returns the accepted state, never a second application. The server alone decides timeouts.

Layer 2The mechanism

The socket tier holds WebSocket connections (RFC 6455) and is separate from game logic, so deploying game code does not drop a million sockets. Lichess has the same split: the main Scala application, lila, and a separate WebSocket server, lila-ws, which talks to it over Redis (lila-ws). Inside the game actor, legality uses bitboards: 64-bit integers per piece type and color, with precomputed attack tables, so generating legal moves for a position takes on the order of a microsecond. Threefold repetition uses Zobrist hashing: each (piece, square) pair has a random 64-bit key, the position hash is the XOR of the present pieces' keys plus side-to-move and castling keys, and each move updates it with two or three XORs. The actor keeps the hashes since the last capture or pawn move and counts matches. Know which rules are automatic: under FIDE Laws, threefold repetition and the 50-move rule are draws a player must claim, while fivefold repetition and 75 moves without a pawn move or capture end the game automatically (FIDE Laws, article 9). Sites choose whether to auto-claim the softer ones.

Layer 3The machine

A move message is about 60 bytes of JSON in a WebSocket frame with a 2-byte header from the server (clients must add a 4-byte mask, RFC 6455, section 5.3), well inside one TCP segment. Socket servers set TCP_NODELAY, because Nagle's algorithm combined with delayed ACKs can hold a small write for tens of milliseconds, which in bullet chess is visible. One epoll_wait loop thread can multiplex tens of thousands of idle connections; most of the per-connection cost is socket buffers in the kernel. A live game's state is about 1 to 2 KB, so 200,000 live games are about 400 MB of RAM, which a handful of game servers hold easily. Processing a move (ply check, clock arithmetic, legality, Zobrist update) takes a few microseconds of CPU. The dominant costs are the 1 ms log append and the network: a transatlantic round trip is about 70 to 90 ms, which is why lag compensation exists at all.

Why it's built this way

  1. A move must be applied exactly once and in order, so one process owns each game and serializes its messages.
  2. In-memory state is lost on a crash, so each accepted ply is appended to a replicated log before anyone sees it.
  3. Networks add tens of milliseconds that differ by player, and bullet games are decided by milliseconds, so the server measures time and compensates for lag up to a cap.
  4. Socket connections are numerous and fragile, while game logic changes often, so they live in separate tiers.
  5. Ratings, archiving and cheat detection are slow, so they consume the end-of-game event asynchronously.

Failure modes

In the wild

Trade-offs

ChoiceStrengthsWeaknessesChoose when
Stateless service plus database per moveSimple, crash-safeDatabase round trip per move, races need retriesCorrespondence chess, low volume
Actor per game in memory plus move logMicrosecond logic, natural orderingOwnership and failover must be engineeredReal-time play (chosen)
Client-side clocksNo server timersTrivially cheatableNever for rated play
Server clocks with capped lag creditFair across connectionsLag estimate can be gamed within the capOnline play at any speed
Match on arrivalLowest latency when busyPoor pairs when quietVery large pools only
Batch matching with widening windowsBetter pairs, simple to scale per poolUp to a cycle of extra waitGeneral matchmaking (chosen)

Leadership lens

Check yourself

  1. A game server pauses for a long GC and wakes up still believing it owns its games. What stops it from forking a game?

    Show answer

    Ownership is a lease, and the move log append carries a fencing token, so writes from the stale owner are rejected. The server is also meant to stop accepting moves before its lease expires, checked against a local monotonic clock with a safety margin.

  2. Why does the server cap the lag credit it gives a move instead of trusting the client's reported transit time?

    Show answer

    Without compensation, a player on a 150 ms round trip loses about 3 seconds over a 40-move bullet game, but a client that claims huge lag would otherwise get free time. So the credit is capped from the player's ping history, never above a fixed maximum such as 500 ms, accepting that the estimate can be gamed within the cap.

  3. Why use an in-memory actor per game plus a move log rather than a stateless service that reads and writes the database on every move?

    Show answer

    The actor processes one message at a time, so there are no locks or races and move logic costs microseconds, while the stateless option pays a database round trip per move and needs optimistic retries. The price is that ownership and failover must be engineered, using leases and replaying the replicated move log after a crash.

  4. Estimate the storage per finished game and explain why bandwidth is not the hard resource.

    Show answer

    80 plies at 4 bytes (2 for the move, 2 for the clock) is 320 bytes plus about 200 bytes of metadata, about 0.5 KB per game or 5 GB a day. Move traffic at 50,000 moves per second to six recipients is only about 30 MB/s; the hard resource is about 1 million open connections at 10 to 50 KB each.

Sources and further study

Primary

Systems docs

Books

Other curricula

Watch

Free resources

Open in the interactive app: practice, quiz, mock interview →