Engineering Atlas › System Design Interviews

Taming hot keys and hot partitions

13 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

How it works

Why hashing does not save you

Consistent hashing or range partitioning gives each shard a fair share of the key space. Real traffic follows a power law, so a handful of keys carry a large fraction of requests: traces from 153 of Twitter's production cache clusters showed approximately Zipfian popularity, sometimes with very high skew (Yang et al., OSDI 2020). If one key gets 200,000 reads per second and a single Redis shard comfortably serves something on the order of 100,000 simple operations per second per core, that shard saturates no matter how many other shards you add. Adding nodes helps the long tail; it does nothing for the head.

Read-hot keys

The goal is to serve one logical key from many physical places.

Write-hot keys

Copying does not help writes, because every copy must take every write. Instead:

Detection

Pre-mark predictable cases such as known celebrities; for the rest, count. Each shard or proxy keeps a count-min sketch or a Space-Saving top-k structure over a sliding window, reports keys above a threshold, and a control plane flips them into "hot mode" (enable salting, extend TTLs, push to L1). Redis offers redis-cli --hotkeys, which works only when maxmemory-policy is an LFU policy, because it reads the eviction counters (redis-cli docs). Since Redis 8.6 the server can also track hot keys itself: HOTKEYS START records the top K keys by share of CPU time or network bytes over a window, with optional sampling and per-slot filters (Redis docs). Managed stores such as DynamoDB shift capacity toward hot partitions and can isolate one heavily used item on its own partition, but that item still tops out at 3,000 read units and 1,000 write units per second (AWS docs).

How it shows up in interviews

Questions that trigger it: design Twitter or Instagram (celebrity posts), design a like or view counter, design a ticketing system, design a distributed cache, design a leaderboard, design a rate limiter keyed by API key. A strong answer separates read-hot from write-hot, sizes the problem with numbers, picks a mechanism for each, and mentions detection. Interviewers then probe: how do you invalidate eight copies consistently, what does a split counter cost on read, what if the hot key is not known in advance, and what if the hot key is a write that needs strong consistency.

Down the stack

Layer 1The contract

A salted read-replica scheme promises that a read returns a value no older than the replication or invalidation delay, typically bounded by a TTL you choose (for example 2 seconds). A split counter promises that the sum is eventually exact once in-flight increments land, with no lost increments as long as each sub-key write is durable; a read during writes may undercount by in-flight operations. A local L1 cache promises nothing about cross-server consistency: two users on different servers can see different values for up to the TTL.

Layer 2The mechanism

In Memcached at Facebook, a miss on a hot key can cause a thundering herd of database queries. Their published design uses leases: on a miss, the server hands out a 64-bit lease token to one client and tells others to wait briefly and retry; the paper describes issuing a token at most once every 10 seconds per key, which cut peak database queries on such keys from 17,000 to 1,300 per second. The token also rejects stale sets if the key was deleted meanwhile (Nishtala et al., NSDI 2013). Redis client-side caching uses CLIENT TRACKING: the server remembers which keys each connection read and sends invalidation messages (over RESP3 push or a redirected Pub/Sub channel) when they change, so an L1 cache can hold values without a fixed TTL. A count-min sketch keeps d rows of w counters; an increment bumps one counter per row and the estimate is the row minimum, which can overestimate but never underestimates (Cormode and Muthukrishnan). Redis's built-in Top-K type pairs such counters with a min-heap of the K heaviest items, using the HeavyKeeper algorithm, whose decay favours large flows over small ones (Redis docs).

Layer 3The machine

A hot Redis key is a hot core: Redis executes commands on one main thread, so one key's operations serialize on one CPU regardless of how many cores the box has. Bandwidth is often the real wall. A 100 KB value read 50,000 times per second is 5 GB/s, about 40 Gbit/s, more than a 25 Gbit NIC can push, so even a perfect CPU path fails. In-process, a single shared counter updated by many threads bounces one 64-byte cache line between cores; each transfer costs on the order of tens to a hundred nanoseconds, which is why LongAdder in Java (Java docs) and per-CPU counters in the Linux kernel stripe a counter across cells, the same split-counter idea at nanosecond scale.

Why it's built this way

Hashing spreads keys evenly because it ignores popularity, so popularity has to be handled separately. A key lives on one owner so that writes have a single serialization point, so read scaling requires copies and write scaling requires splitting the key itself. Copies go stale, so every copy scheme picks a staleness bound (TTL, lease, invalidation). Counting every key exactly costs memory proportional to key count, so detection uses sketches with bounded error.

Diagram

flowchart LR
    C["Clients"] --> A["App server with L1 cache"]
    A -->|"miss, key not hot"| S1["Shard owning post:42"]
    A -->|"key hot: pick #0..#7"| R["Salted copies on 8 shards"]
    S1 --> D["Hot key detector (top-k sketch)"]
    D -->|"mark hot"| CP["Control plane"]
    CP -->|"enable salting and L1"| A

Worked example

A social app shows like counts on posts. Normal posts get tens of likes per second. A celebrity post receives 60,000 likes per second and 400,000 count reads per second at peak.

Writes: one Redis shard handles perhaps 80,000 increments per second in practice, and the hot shard also serves other keys. Split the counter into 32 sub-keys spread across shards: each gets about 1,900 increments per second. Add write buffering on 300 app servers with a 500 ms flush: each server sees 200 likes per second, flushes 100 per batch, so the cluster receives about 600 INCRBY calls per second total. A crash loses up to 500 ms of likes on one server: fine for likes, not for payments.

Reads: summing 32 sub-keys per read at 400,000 reads per second would be 12.8 million Redis operations per second. Instead, a background job sums the sub-keys every second and writes likes:42:total. App servers cache that total in L1 for 1 second. With 300 servers, the total key sees at most 300 reads per second. Counts are up to about 2 seconds stale.

In the wild

Trade-offs

OptionStrengthsWeaknessesChoose when
Local L1 cacheRemoves load entirely, microsecond readsStale per server, memory per instanceRead-hot, staleness of seconds is fine
Salted key copiesRead capacity scales with copiesN writes or invalidations per updateRead-hot, values change rarely
Split counterWrite capacity scales with splitsReads cost N lookups, needs a cached sumWrite-hot commutative updates
Write bufferingHuge reduction in write rateLoses buffered data on crash, adds delayCounters, metrics, analytics
Single owner queueExact, strongly consistentThroughput capped by one workerInventory, money, anything non-commutative

Pitfalls in production

Leadership lens

Check yourself

  1. Why does adding more shards not fix a hot key?

    Show answer

    All requests for one key still route to the single shard that owns it, so extra shards only take load from other keys.

  2. Why do salted copies help reads but not writes?

    Show answer

    Readers can pick any copy, spreading reads, but every write must reach every copy, so write load multiplies instead of spreading.

  3. A split counter has 16 sub-keys. What does a read cost, and how do you keep reads cheap?

    Show answer

    Sixteen lookups per read; keep a periodically recomputed cached total so most reads hit one key.

  4. Why is a count-min sketch suitable for hot-key detection?

    Show answer

    It uses fixed memory regardless of key count and only overestimates, so heavy hitters are never missed.

Sources and further study

Primary

Systems docs

Books

Other curricula

Watch

Free resources

Open in the interactive app: practice, quiz, mock interview →