Engineering Atlas › System Design Interviews

Cassandra for system design interviews

13 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

How it works

Data model

A table has a partition key and optional clustering columns. The partition key decides which nodes hold the data; clustering columns sort rows inside the partition on disk. PRIMARY KEY ((sensor_id, day), ts) puts each sensor-day in one partition sorted by time, so "sensor 42 on 1 March after 10:00" is one sequential read. With no joins, you design one table per query and denormalise.

Placement and requests

The Murmur3 partitioner hashes the partition key to a 64-bit token. Nodes own several token ranges (vnodes, num_tokens, 16 by default in current releases, down from the 256 that 2.x's random allocation needed), and NetworkTopologyStrategy picks replicas on distinct racks in each data centre. A token-aware driver sends each request to a coordinator, which forwards to all replicas and waits for as many as the consistency level demands: ONE, LOCAL_QUORUM, QUORUM, EACH_QUORUM or ALL. Every replica still receives the write; the level only sets how many must answer first.

Convergence

If a replica is down during a write, the coordinator stores a hint and replays it when the node returns, within max_hint_window (3 hours by default) (cassandra.yaml reference). Reads that see disagreeing replicas trigger read repair. Older divergence needs nodetool repair, which compares Merkle trees of token ranges and streams differences.

Down the stack

Layer 1The contract

With W + R > RF, a successful read returns the latest successfully acknowledged write, provided clocks are sane, because the write set and read set share a replica. A write that fails to reach its consistency level returns a timeout or UnavailableException, but may still have been applied on some replicas; it is neither committed nor rolled back, so clients must retry idempotently. Conflicts resolve per cell by write timestamp, with deletions winning ties. Lightweight transactions (IF NOT EXISTS, IF col = x) use Paxos for linearizable compare-and-set on one partition. The default paxos_variant: v1 costs four round trips per write and three per read; the newer v2, which the config reference recommends, needs two for a write and one or two for a read (cassandra.yaml reference). Typical latency is 1 to 5 ms at p99 for single-partition LOCAL_QUORUM operations on a healthy cluster (hedge: heavily depends on partition size and GC).

Layer 2The mechanism

Write path. The replica appends the mutation to the commit log, inserts it into the memtable (a sorted in-memory structure per table) and acknowledges. When a memtable fills, or the commit log nears its size limit, it is flushed to a new SSTable and the commit log segments it covered are recycled (Cassandra docs). commitlog_sync: periodic (default, fsync every 10 seconds via commitlog_sync_period) acknowledges before the data is on disk, relying on replication for durability; batch or group modes wait for the fsync. If every replica loses power inside the same window, acknowledged writes from that window are gone.

SSTable layout. Each SSTable is a set of immutable files: Data.db (partitions in token order, rows in clustering order, compressed in chunks, 16 KB by default), Index.db and Summary.db (partition key to offset, with an in-memory summary sampling every 128th index entry) in the classic big format (still the default), or trie-based partition and row indexes (Partitions.db, Rows.db) in the bti format added in 5.0 and enabled with sstable: selected_format: bti (Cassandra docs), Filter.db (a Bloom filter over partition keys), CompressionInfo.db (chunk offsets), Statistics.db (min and max timestamps, tombstone counts) and TOC.txt.

Read path. For each SSTable, the Bloom filter says "definitely not here" or "maybe". For maybes, the key cache or partition index gives an offset, CompressionInfo maps it to a compressed chunk, and the chunk is read and decompressed. Results from the memtable and every relevant SSTable are merged, newest timestamp per cell winning, and tombstones suppress older data.

Coordinator read and read repair. At QUORUM with RF 3, the coordinator sends a full data request to the closest replica and digest requests (a hash of the result) to one other. If the digests match, it returns. If not, it requests full data from those replicas, resolves per cell by timestamp, returns the winner, and sends repair mutations to the stale replicas, blocking until they acknowledge so that a later quorum read cannot go backwards. Since 4.0 this is the per-table read_repair = 'BLOCKING' default; NONE keeps partition-level write atomicity but gives up monotonic quorum reads, and the old background read_repair_chance is gone (Cassandra docs).

Compaction. Compaction merges SSTables, dropping overwritten cells and tombstones older than gc_grace_seconds (10 days by default). Size-tiered (STCS) is write-friendly; leveled (LCS) keeps a partition in about one SSTable per level for reads at more write amplification; time-window (TWCS) buckets by time so TTL'd data drops as whole files; unified (UCS, 5.0) spans these. Cassandra 5.0, the current release line, also added trie memtables, Storage Attached Indexes (SAI) and a vector type (Cassandra 5.0: what's new).

Gossip and failure detection. Every second each node bumps its heartbeat and gossips with one random live peer, sometimes with an unreachable one, and with a seed if the random pick was not a seed, exchanging heartbeat generations and versions (Cassandra docs). A phi accrual failure detector turns heartbeat inter-arrival times into a suspicion level and marks a peer down above a threshold, which makes coordinators stop sending it requests and start writing hints.

Layer 3The machine

A write is one sequential append to the commit log on disk and a memory insert; no random I/O happens on the write path, which is why one node can absorb tens of thousands of writes per second (hedge: roughly 5,000 to 15,000 per core for small rows). Reads are where I/O goes: with leveled compaction a point read touches about one SSTable, so one or two 4 KB to 16 KB reads on SSD at 100 us each; with size-tiered compaction on a partition updated for months it might touch ten.

Cassandra runs on the JVM, with requests flowing through per-stage thread pools (native transport, mutation, read) and Netty handling sockets. Garbage collection is the main tail-latency risk: large partitions or wide IN queries churn objects, and a long pause makes the node look dead to gossip. Teams use G1 or ZGC with 8 to 31 GB heaps (staying under 32 GB keeps compressed object pointers), and leave the rest of RAM to the page cache, which is what actually serves hot SSTable chunks. Compaction needs free disk: size-tiered can temporarily need as much free space as the SSTables being merged, which is why operators keep disks under about 50 percent full.

Why it's built this way

  1. Random disk writes are slow and sequential ones are fast, so writes go to an append-only log and sorted immutable files, an LSM tree.
  2. Immutable files accumulate and slow reads, so compaction merges them in the background, and Bloom filters skip files that cannot contain a key.
  3. Availability across data centres mattered more than single-copy consistency, so there is no leader; any replica accepts writes and timestamps decide conflicts.
  4. Leaderless replicas diverge, so hints, read repair and Merkle-tree repair converge them at increasing cost and coverage.
  5. Deletes in an append-only system must be written as tombstones, and tombstones must outlive the longest repair interval, hence gc_grace_seconds.

Diagram

sequenceDiagram
    participant C as Client
    participant Co as Coordinator
    participant A as Replica A
    participant B as Replica B
    C->>Co: SELECT at QUORUM (RF 3)
    Co->>A: data request
    Co->>B: digest request
    A-->>Co: row (ts 200)
    B-->>Co: digest mismatch
    Co->>B: full data request
    B-->>Co: row (ts 150)
    Co->>Co: newest timestamp wins (A)
    Co->>B: repair mutation (ts 200)
    B-->>Co: ack
    Co-->>C: row from A

Worked example

A fleet of 2 million IoT devices sends a 200-byte reading every 10 seconds: 200,000 writes per second, about 40 MB/s of raw data, 3.5 TB per day before replication.

The table is PRIMARY KEY ((device_id, day), ts) with TWCS using one-day windows and a 30-day TTL. Partitions hold one device-day, 8,640 rows of about 200 bytes, so under 2 MB, comfortably below the 100 MB guideline. With RF 3, the cluster absorbs 600,000 replica writes per second. At a conservative 20,000 writes per second per node, that is 30 nodes for throughput.

Storage: 3.5 TB per day times 30 days times RF 3 is 315 TB raw, perhaps 100 TB after LZ4 (hedge: ratios vary). TWCS expires whole files, so nodes can run 60 percent full; at 4 TB disks that is about 45 nodes, so storage, not writes, sets the size. "Last hour for device X" at LOCAL_QUORUM reads one partition from a few SSTables, around 2 to 5 ms.

In the wild

Trade-offs

OptionStrengthsWeaknessesChoose when
Cassandra or ScyllaDBLinear write scaling, multi-DC active-active, no single leaderQuery-first modelling, tombstone and repair operations, no joinsWrite-heavy, multi-region, known access patterns
DynamoDBFully managed, predictable latencyAWS-only, per-partition throughput ceilings, cost at very high steady loadSame shape of workload, team without database ops
PostgresSQL, transactions, flexible queriesSingle-writer node, harder to go multi-regionRelational data at moderate write rates
HBase or BigtableStrong per-row consistency, range scansSingle region-server per range, HDFS or managed onlyOrdered scans over huge tables

Pitfalls in production

Leadership lens

Check yourself

  1. With RF 3, why do QUORUM writes plus QUORUM reads give read-your-writes, but ONE plus ONE does not?

    Show answer

    Two out of three plus two out of three must share a replica, so the read sees the newest write; one plus one can hit different replicas.

  2. A deleted row reappears weeks later. What happened?

    Show answer

    A replica missed the tombstone, the others purged it after gc_grace_seconds, and later repair or reads propagated the old live data back.

  3. Why does a read at QUORUM block on read repair?

    Show answer

    So that once a quorum has returned a value, any later quorum read sees it too; otherwise successive reads could return new then old.

  4. Why use TWCS for TTL'd time series?

    Show answer

    Each SSTable holds one time window, so once all data in it expires, the whole file is dropped without merging or scanning tombstones.

Sources and further study

Primary

Systems docs

Books

Other curricula

Watch

Free resources

Open in the interactive app: practice, quiz, mock interview →