Engineering Atlas › System Design Interviews

Design a real-time game ranking board

21 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

Requirements

Functional:

Non-functional:

Out of scope: matchmaking, anti-cheat detection models (we consume their verdicts), rewards payout, and the game client.

Back-of-envelope

Assume a popular mobile game with 50 million daily active users and 120 million players with a season score.

Writes: each active player finishes about 10 matches a day, so 500 million score events per day. 500,000,000 / 86,400 is about 5,800 per second on average. Evening peaks and weekend events run about 5x, so plan for 30,000 per second. Each event updates three windows (daily, weekly, season), so the cache sees about 90,000 sorted-set writes per second at peak. A single Redis process does roughly 100,000 to 200,000 simple commands per second, which is close to the limit, so we will need at least a few shards by mode or window.

Reads: each player opens the leaderboard screen about 5 times a day, and each screen issues one top-100 read, one rank read and one neighbourhood read. 50 million x 5 x 3 is 750 million reads a day, about 8,700 per second on average and 45,000 at peak. Top-100 is the same for everyone, so a 1-second cached snapshot removes a third of that load.

Memory: a sorted-set member costs roughly 100 to 130 bytes (skip list node, hash table entry, a 16-byte player ID string, allocator rounding). 120 million x 120 bytes is about 14 GB for the season board. Daily and weekly boards hold about 50 and 80 million active players, roughly 6 GB and 10 GB. That is about 30 GB per game mode, or 150 GB for five modes before replicas.

Durable log: a score event is about 100 bytes (match ID, player ID, mode, score, timestamp, server signature). 500 million x 100 bytes is 50 GB per day, about 18 TB per year, and 90 TB over five years uncompressed. Columnar compression brings that to perhaps 15 to 20 TB. The current-best table (one row per player, mode and window) is small by comparison: a few hundred million rows of about 60 bytes, tens of GB.

Bandwidth: a top-100 response of about 8 KB at 15,000 per second is 120 MB/s, almost all served from cache. Score submissions are tiny.

API

POST /v1/scores                     (game server only, mTLS + signed payload)
  body { match_id, player_id, mode, score, ended_at }
  -> 202 { accepted: true }          (duplicate match_id returns the same 202)

GET  /v1/boards/{mode}/{window}/top?limit=100
  -> { as_of, entries: [ { rank, player_id, name, score } ] }

GET  /v1/boards/{mode}/{window}/players/{player_id}
  -> { rank, score, percentile, above: [ ... 10 ], below: [ ... 10 ] }

GET  /v1/boards/{mode}/{window}/friends?player_id=...
  -> { entries: [ { rank_among_friends, player_id, score } ] }

DELETE /v1/boards/{mode}/players/{player_id}     (operator; removes from all windows)
POST   /v1/boards/{mode}/{window}/rebuild        (operator; replays the log)

window is a concrete key such as daily-2026-10-16, weekly-2026-W42 or season-14, never "current", so clients and caches agree on which board they are looking at around midnight.

Data model

score_events (append-only log, Kafka topic partitioned by player_id, archived to object storage as Parquet): match_id, player_id, mode, score, ended_at, server_id, signature. Partitioning by player keeps all of one player's events in order on one partition, so the consumer can apply "keep best" without cross-partition races.

player_best (wide-column store such as Cassandra or DynamoDB, partition key (mode, window, player_id)): best_score, achieved_at, match_id. This is the durable truth of each board. Updates are conditional (IF best_score < :new in Cassandra, or a DynamoDB condition expression such as best_score < :new (DynamoDB docs)), so replays are harmless. A board rebuild is a full scan of one (mode, window) prefix, so in DynamoDB we would add a sharded secondary key (mode, window, bucket 0..255) to scan it in parallel.

processed_matches (same store, partition key match_id, TTL 30 days): makes submission idempotent. Insert-if-absent before producing to the log.

Redis sorted sets (key lb:{mode}:{window}, member player_id, score = encoded sort key): the serving index. Each board is one key, so boards spread across cluster slots independently.

friends (owned by the social service): read at query time, at most a few hundred IDs per player.

High-level design

flowchart LR
  GS["Game servers"] --> ING["Score ingest API"]
  ING --> DED[("processed_matches")]
  ING --> LOG[["Kafka: score_events"]]
  LOG --> UPD["Board updater"]
  UPD --> BEST[("player_best")]
  UPD --> RZ[("Redis sorted sets")]
  LOG --> ARC[("Object storage archive")]
  AC["Anti-cheat verdicts"] --> UPD
  C["Game client"] --> API["Leaderboard API"]
  API --> SNAP["Top-N snapshot cache"]
  API --> RZ
  API --> SOC["Social graph"]
  SNAP --> RZ

Write path: the game server, not the client, posts the final score when a match ends. Clients are never trusted to report scores. The ingest API checks the signature, inserts match_id into processed_matches with insert-if-absent, and on success produces the event to Kafka with acks=all. A duplicate returns 202 without producing. The board updater consumes in per-player order, computes the new best for each affected window, writes player_best conditionally, and if the condition held, issues ZADD lb:{mode}:{window} GT <key> <player> for each window. The GT flag, available since Redis 6.2 (Redis docs), only raises a member's score, so a late or replayed event can never lower it. Kafka offsets are committed after the Redis writes, so a crash replays a few events, which the conditional writes absorb.

Read path: top-100 comes from a snapshot the API refreshes every second with ZRANGE lb:... 0 99 REV WITHSCORES, decorated with display names from a profile cache. "My rank" is one O(log n) command since Redis 7.2: ZREVRANK lb:... <player> WITHSCORE returns the rank and the score together (Redis docs). On older servers, pipeline ZREVRANK with ZSCORE, which costs the same round trip but can straddle a write. The neighbourhood is ZRANGE lb:... (rank-10) (rank+10) REV. A friends board fetches the friend list and runs a pipelined ZMSCORE for those IDs, then sorts at most a few hundred entries in the API process. For read-your-writes, the ingest response returns the new best, and the client shows it locally until the board catches up.

Deep dives

Ranking at 100 million players

Option one is a relational table indexed on score with SELECT COUNT(*) WHERE score > :mine. Ordinary B-trees keep no subtree counts, so the 50-millionth player's query reads 50 million index entries.

Option two is the sorted set. Redis keeps each large sorted set as a hash table (Redis docs) (member to score, for O(1) ZSCORE) plus a skip list ordered by (score, member). This is the skip list of Pugh (1990) with one addition: every forward pointer stores a span: how many level-0 nodes it jumps over. Walking down from the top level and summing spans gives the rank of any node in O(log n) expected steps. This is an order-statistic structure, and it is why ZREVRANK is fast where SQL is not. See Skip Lists and Redis for the internals.

Option three, for boards that outgrow a node, is sharding. By player ID breaks rank, since every lookup must fan out and sum "players above you" across all shards. By score range keeps rank cheap ("players in higher shards plus your rank in your shard"), but skewed, drifting score distributions force rebalancing. Option four drops exactness below the top: exact sorted sets for the top 100,000 and a histogram of counts per score bucket for everyone else, queried through a Fenwick tree in O(log buckets). Counts add, so the histogram shards and merges trivially.

Choice: one sorted set per board while it fits (it does at 14 GB), with the histogram maintained alongside so the API can serve "top 4%" if a board ever has to be split. The practice task builds that histogram index.

Ties, score encoding and windows

Two players with 9,000 points must not swap places on every refresh. The common rule is that whoever reached the score first ranks higher. A sorted set orders equal scores lexicographically by member string (Redis docs), which is arbitrary for player IDs, so encode the tie-break into the score itself. Redis scores are IEEE 754 doubles, exact for every integer up to 2^53 (Wikipedia). With game scores below 2^21 (about 2 million) and seconds within a season below 2^31, the key score * 2^31 + (2^31 - 1 - seconds_since_season_start) is an exact integer under 2^52, and larger means better on both fields. Decode by integer division when reading. If scores can exceed that, give the tie-break fewer bits (minutes instead of seconds) or move to a store with composite keys.

Windows are separate keys. The updater writes daily, weekly and season keys for each event, and each key gets EXPIRE at window end plus a grace period. A sorted set keeps no per-member history, so "this week" cannot be computed at read time from one board. Resets are free: a new key starts empty. A late event goes to the window of ended_at, not arrival time, while that key still exists.

Durability, rebuilds and cheaters

Redis is in memory. With replication and AOF it survives most failures, but the board must be rebuildable from something we trust more: the event log and player_best. A rebuild streams player_best rows for the (mode, window) into a new key lb:...:v2 with pipelined ZADD batches of 1,000, then swaps the key name with RENAME, so readers never see a half-built board. At 1 million members per second per pipeline, a 120-million-member board rebuilds in about two minutes, with several parallel loaders writing to the same key.

When anti-cheat flags a player, the updater runs ZREM on every window key and marks the player in player_best. Ranks are computed, not stored, so everyone below moves up by one with no rewrite. If instead the scoring formula itself was wrong, we replay the archived log through the fixed formula into fresh keys, which is the event sourcing argument applied to a game.

Down the stack

Layer 1The contract

For a key lb:{mode}:{window}, after POST /scores returns 202 for match M, the board eventually reflects max(previous best, M.score) for that player, and no replay of M changes it again. Rank is defined as 1 plus the number of members whose encoded key is strictly greater, so equal raw scores are ordered by who got there first, and the order is total. Reads are consistent per command (one ZREVRANK sees one point in time) but a pipelined rank plus neighbourhood read may straddle a write; the API treats that as acceptable. Staleness from submission to board is bounded by consumer lag, with an alert at 2 seconds.

Layer 2The mechanism

In Redis's t_zset.c, a sorted set with more than 128 members (the default zset-max-listpack-entries) or any member longer than 64 bytes is converted from a compact listpack to the zset encoding: a dict plus a zskiplist. Each zskiplistNode holds the member SDS, a double score, a backward pointer and a flexible array of levels, each with a forward pointer and an unsigned long span. A node's level grows with probability 0.25 per step, capped at 32 (ZSKIPLIST_P and ZSKIPLIST_MAXLEVEL in server.h), so it averages 1.33 levels. ZADD ... GT reads the old score from the dict and, if the new one is larger, moves the node and fixes spans along the update path. ZREVRANK descends from the top level summing spans and returns length - rank. Replicas receive the command stream in order, and AOF with appendfsync everysec fsyncs once per second from a background thread, so a crash can lose about a second of writes (Redis persistence docs).

Layer 3The machine

A skip list search over 120 million members descends about 13 levels (log base 4), with a few hops per level: 30 to 50 pointer dereferences, mostly cache misses at 80 to 100 ns each, so 3 to 5 microseconds per rank lookup. Request parsing costs about as much, and the in-zone network round trip of 200 to 500 microseconds dominates what the API sees. The expensive moment is persistence: an RDB snapshot or AOF rewrite calls fork() on the main thread, and the kernel copies page tables for 14 GB of resident memory (about 28 MB of entries with 4 KB pages). Redis's own measurements put that at roughly 9 to 13 ms per GB on physical machines and VMware, so about 130 to 180 ms of stalled event loop for this board, and 239 ms per GB on old Xen-based EC2 instances, over 3 seconds (Redis latency docs). The same page says to disable transparent huge pages, which otherwise turn copy-on-write into copying almost the whole process. Copy-on-write then duplicates every page the updater touches, so leave 30 to 50% RAM headroom and snapshot on replicas.

Why it's built this way

  1. Rank queries need "how many above me" in logarithmic time, so we need a structure that stores counts on its internal links; ordinary database indexes do not, so the serving index lives in a specialised in-memory store.
  2. Memory is volatile and expensive, so the in-memory board is a derived view, and the durable truth is a compact per-player best table plus an append-only log.
  3. Logs and networks deliver duplicates and late events, so every write is idempotent (insert-if-absent on match ID) and monotone (GT, conditional best update), which makes replays safe.
  4. One process has a finite write rate and RAM, so boards are split by mode and window first (natural, independent keys) and by score only when a single board outgrows a node.
  5. Players care about exact rank only near the top, so approximate rank from bucket counts is an acceptable escape hatch that scales by addition.

Failure modes

In the wild

Trade-offs

OptionStrengthsWeaknessesChoose when
SQL table with score indexSimple, durable, transactionalRank needs a count over all rows above youUnder about a million players, or rank shown only for the top
One Redis sorted set per boardO(log n) rank and range, exact, operationally familiarBounded by one node's RAM and single-threaded writes; needs a durable source of truthBoards up to roughly 100 to 200 million members
Score-range sharded sorted setsExact rank beyond one nodeRebalancing as the distribution shifts; two calls per rank lookupA single board too big for one node and exact rank is a product requirement
Bucket histogram (Fenwick over counts)Tiny memory, trivially sharded and merged, O(log buckets)Rank is approximate inside a bucketLong tail of players, "top 3%" is acceptable
Batch ranking in a warehouseCheap, handles any formulaMinutes to hours staleEnd-of-season final standings and rewards

Leadership lens

Check yourself

  1. Why can a Redis sorted set answer "what is my rank" in logarithmic time when SELECT COUNT(*) WHERE score > :mine on an indexed SQL table cannot?

    Show answer

    Each forward pointer in the skip list stores a span, the number of level-0 nodes it jumps over, so summing spans on the way down gives the rank in O(log n) steps. Ordinary B-trees keep no subtree counts, so the SQL query has to read every index entry above you, 50 million entries for the 50-millionth player.

  2. The Redis primary for a board is lost and the promoted replica missed the last few hundred milliseconds of writes. Why are no accepted scores lost?

    Show answer

    Redis is a derived view; the durable truth is the Kafka log and player_best. The updater commits Kafka offsets only after its Redis writes, so it re-consumes from a point behind the lost writes, and conditional best updates plus ZADD GT make the replayed events harmless.

  3. Using the lesson's estimates, why does one season board fit on a single node, and why do you still expect to need a few Redis shards?

    Show answer

    120 million members at about 120 bytes each is roughly 14 GB, which fits in one node's memory. But 30,000 score events per second at peak, each updating three windows, is about 90,000 sorted-set writes per second, close to a single Redis process's 100,000 to 200,000 command limit, so boards are split by mode or window.

  4. When a single board outgrows one node, what are the trade-offs between sharding by player ID, by score range, and switching to a bucket histogram?

    Show answer

    Sharding by player ID breaks rank, since every lookup must fan out and sum players above you across all shards. Score-range shards keep exact rank cheap but need rebalancing as the score distribution drifts, while a Fenwick tree over score buckets is tiny and merges by addition but gives only approximate rank inside a bucket.

Sources and further study

Primary

Systems docs

Books

Other curricula

Watch

Free resources

Open in the interactive app: practice, quiz, mock interview →