MongoDB for system design interviews
Open in the interactive app: practice, quiz, mock interview →In one minute
- MongoDB stores BSON documents in collections, with secondary indexes, aggregation pipelines and multi-document transactions. One document may be at most 16 MiB.
- Each shard is a replica set: a primary takes writes, secondaries pull its oplog, and a Raft-style protocol with terms elects a new primary in roughly ten seconds.
- Durability and freshness are per-operation dials: write concern (
w: 1,w: "majority") and read concern (local,majority,linearizable,snapshot). - WiredTiger stores each collection and index as a B-tree with MVCC, a write-ahead journal and a checkpoint every 60 seconds.
- Sharding splits a collection into key ranges (chunks) placed by a balancer and routed by
mongos. The shard key is the decision you cannot cheaply undo.
How it works
The document model
A document holds what is read together: an order with its line items and address. One read fetches the aggregate, and a single-document write is atomic without a transaction. Unbounded arrays (every comment on a viral post) fight back: they grow towards 16 MiB, rewrite large pages on each update and bloat indexes, so anything that grows without bound gets its own collection.
Replica sets
A replica set has up to 50 members, at most 7 voting (MongoDB docs). The primary applies a write and records it in the oplog (local.oplog.rs); secondaries copy and replay it. Drivers discover the topology and send reads where the read preference allows (primary, secondary, nearest).
Sharded clusters
Stateless mongos routers consult a config server replica set that maps key ranges to shards. A query with the shard key hits one shard; without it, every shard. Hashed keys spread inserts but scatter range queries; ranged keys keep ranges local but funnel monotonic keys (timestamps, ObjectIds) onto one hot shard. Since 8.0 the topology is less fixed than it was: moveCollection relocates an unsharded collection to another shard, unshardCollection gathers a sharded one back onto a single shard, a config server can also hold data as a config shard, and resharding runs substantially faster, including resharding on the same key to redistribute data (MongoDB 8.0 release notes).
Down the stack
Layer 1The contract
A single-document write is atomic and, on the primary, linearizable. With w: "majority" (the implicit default for most deployments since 5.0), an acknowledged write is journaled on a majority of voting members and survives any failover the majority survives. The exception catches people out: with an arbiter, if data-bearing voters do not outnumber the voting majority (the common primary, secondary, arbiter layout), the default silently drops to w: 1 (MongoDB docs). With w: 1, only the primary had it, and if it dies first the write is rolled back when it rejoins. readConcern: "majority" never returns data that can be rolled back; local may. Causally consistent sessions add read-your-writes across members by carrying afterClusterTime, but only with majority reads and writes; with w: 1 and local reads, Jepsen saw acknowledged writes vanish and reads return values later rolled back (Jepsen). Multi-document transactions give snapshot isolation, not serializability, and set their own concerns: Jepsen found 4.2.6 transactions ignored collection-level concerns and fell back to local and w: 1, so set snapshot and majority explicitly on each transaction (Jepsen). Expect single-digit millisecond indexed reads within a zone and about 10 to 15 seconds without writes during an unplanned failover; the docs put the median election, including detection, at no more than about 12 seconds with default settings (MongoDB docs).
Layer 2The mechanism
WiredTiger. Each collection and index is a .wt file holding a B-tree. In cache, records carry update chains stamped with transaction timestamps, which gives MVCC snapshot reads without read locks; versions still needed by old readers move to the history store. On disk, pages are compressed (snappy for collections, zstd for time-series collections, prefix compression for indexes) and never overwritten: a reconciled page goes to a new block, and a checkpoint every 60 seconds publishes a consistent root (MongoDB docs). Between checkpoints the journal makes writes recoverable; it is flushed every 100 ms by default (storage.journal.commitIntervalMs), or at once for j: true and majority writes.
Oplog. A capped collection of idempotent operations keyed by optime (timestamp plus term); an $inc is logged as a $set of the result so replay is safe. Secondaries pull with a tailable cursor from a sync source (possibly another secondary), apply batches with parallel writer threads, and report applied and durable optimes with replSetUpdatePosition. The primary computes the majority commit point and gossips it in heartbeats.
Elections. Members heartbeat every two seconds. After electionTimeoutMillis (10 seconds) without the primary, a secondary bumps the term and sends replSetRequestVotes with its last optime. Voters grant one vote per term, only to candidates at least as up to date, Raft's rule. A deposed primary rolls back entries the new primary lacks into a rollback directory.
Balancer. Routers cache the routing table with a collection version; a shard holding a newer version returns a stale-config error and the router refreshes. Since 6.0 the balancer, running on the config server primary, balances by data size: it acts when shards differ by three times the range size (default 128 MB, so 384 MB) (MongoDB docs). A migration clones documents, streams concurrent changes, holds a short critical section that blocks writes to the range, commits ownership on the config servers, and deletes the donor copy later.
Layer 3The machine
The WiredTiger cache defaults to the larger of 50 percent of (RAM minus 1 GB) or 256 MB, so a 64 GB host gets 31.5 GB; the OS page cache holds compressed blocks with the rest. Eviction threads aim to keep the cache near 80 percent full and dirty data near 5 percent; past the hard triggers (roughly 95 percent full or 20 percent dirty), application threads must evict pages themselves and p99 jumps from milliseconds to hundreds of milliseconds. Checkpoints flush all dirty pages each minute, a periodic I/O burst. The journal is append-only with grouped fsyncs: one flush of a few hundred microseconds on NVMe covers every write in the window, so majority writes batch well. Each connection gets its own thread, commonly estimated at around 1 MB of memory, so 20,000 pooled connections hurt before any query runs.
Why it's built this way
- Joins across tables cost random I/O, so the aggregate lives in one document behind one B-tree lookup.
- Single-document writes need no distributed transaction, so data can be sharded by key with each write on one replica set.
- Random in-place updates are slow and torn writes corrupt pages, so WiredTiger writes new blocks, journals sequentially and checkpoints periodically.
- Machines fail, so shards replicate an ordered log, and safe failover needs majorities and terms so two primaries can never both commit.
- A majority costs a round trip, so latency versus durability is chosen per operation.
Diagram
sequenceDiagram
participant C as App (driver)
participant P as Primary
participant S1 as Secondary 1
participant S2 as Secondary 2
C->>P: insert, w majority
P->>P: apply + journal + oplog entry (term 7, ts 105)
S1->>P: getMore on oplog (tailable cursor)
P-->>S1: batch up to ts 105
S1->>S1: apply + journal
S1->>P: replSetUpdatePosition durable 105
P->>P: commit point = 105 (2 of 3)
P-->>C: ok
S2->>P: getMore (lagging)Worked example
An order history service holds 600 million orders of about 2 KB (1.2 TB plus about 300 GB of indexes), with 8,000 inserts and 40,000 reads per second at peak. Nearly every read is "orders for customer X, newest first".
Shard key {customerId: 1, createdAt: -1} makes each read a single-shard index range scan, and inserts spread because customers arrive at random. A hashed _id would spread writes too but scatter every customer read.
Six shards hold about 250 GB each. On 128 GB hosts the cache is about 63 GB, enough for all indexes and the recent orders that dominate reads. Each primary takes about 1,300 inserts per second.
Oplog: at roughly 2.2 KB per entry a shard logs about 2.9 MB per second, 10 GB per hour. With 1.5 TB free disk the default oplog (5 percent of free disk, at least 990 MB and capped at 50 GB) hits its cap: about five hours of window (MongoDB docs). A secondary down longer needs a full initial sync, so before an eight-hour maintenance raise oplogSizeMB or set oplogMinRetentionHours.
Orders use w: "majority" across three zones, adding about 1 to 2 ms to p50 inserts. History pages read from secondaries with maxStalenessSeconds: 90, the minimum allowed.
In the wild
- Foursquare, 2010: one shard outgrows its RAM. Two months after splitting a 66 GB-RAM database into two shards keyed by user ID, uneven check-in activity grew shard0 to about 67 GB while the other held about 50 GB. Once shard0's data no longer fit in memory, queries went to disk, a backlog built and the site went down on two consecutive days. Moving 5 percent of the data to a new shard did not help, because 300-byte documents left the old pages fragmented; a four-hour offline compaction did. 10gen promised online compaction and size-aware balancing, which the balancer now does (MongoDB user group post-mortem).
- Jepsen, 2018: causal sessions without majorities. MongoDB documents read-your-writes and monotonic reads for causal sessions, but with
w: 1writes andlocalreads, acknowledged writes disappeared after a partition healed. MongoDB closed the ticket as working as designed and documented that the guarantees need majority read and write concerns (Jepsen). - Jepsen, 2020: transactions weaker than advertised. In 4.2.6, even
snapshotreads withmajoritywrites showed read skew, cyclic information flow and duplicated writes, and transactions ignored collection-level concerns. MongoDB traced the anomalies to a retry bug (SERVER-48307) and scheduled the fix for 4.2.8 (Jepsen).
Trade-offs
| Option | Strengths | Weaknesses | Choose when |
|---|---|---|---|
w: 1 | Lowest write latency | Acknowledged writes can roll back on failover | Logs, metrics, rebuildable data |
w: "majority" | Survives failover of a minority | One extra intra-cluster round trip | Default for business data |
| Hashed shard key | Even writes, no hot shard on monotonic keys | Range queries scatter to all shards | Point lookups by ID, high insert rate |
| Ranged compound shard key | Targeted range queries | Hot spots if the leading field is skewed or monotonic | Queries scoped by tenant or customer |
| Embed related data | One read, atomic update | Document growth, duplication | Bounded, read-together data |
| Reference in another collection | Unbounded growth, independent updates | Extra query or $lookup | Comments, events, history |
Pitfalls in production
- Monotonic shard key. ObjectId or timestamp keys send every insert to one shard. Hash the key or lead with a high-cardinality field.
- Unbounded arrays. Growing documents rewrite large pages and inflate multikey indexes. Bucket them or move them to their own collection.
- Oplog window too short. A secondary that falls further behind than the window needs initial sync. Alert on lag relative to the window.
- Cache pressure. A working set larger than the cache drafts application threads into eviction. Watch application-thread evictions; add RAM or shards.
- Secondary reads on freshness-critical paths. Lag reaches seconds during bursts. Use primary reads or causal sessions for read-your-writes.
- Connection storms. Huge driver pools swamp thread-per-connection. Cap
maxPoolSize.
Leadership lens
- Treat the shard key as an architecture decision with a written review: resharding exists since 5.0 and is much faster in 8.0, but it still copies the whole collection and needs spare capacity.
- Set write concern by data class in a team standard, and forbid
w: 1for anything that represents money, orders or user content. - Size the oplog by the longest planned outage plus margin, and rehearse failovers to measure real write unavailability.
- Managed Atlas removes upgrade and backup toil; self-hosting needs people who understand elections, balancing and WiredTiger metrics.
- Ask "which queries do not include the shard key?" in every design review; each one costs work on every shard.
Check yourself
A client receives "ok" for a
w: 1insert, the primary crashes, and after failover the document is missing. Where did it go?Show answer
The write never reached a secondary, so the new primary lacked it. When the old primary rejoined it rolled the write back into its
rollbackdirectory.Why does a voter refuse a candidate with an older last optime?
Show answer
A committed write lives on a majority, and any winning majority overlaps it; refusing staler candidates guarantees the new primary holds every committed write.
Why might p99 latency spike while CPU and disk look moderate?
Show answer
The cache crossed its eviction trigger, so application threads are evicting pages before serving their own operations.
Why is
{createdAt: 1}a poor shard key for an insert-heavy collection?Show answer
Every new document has the largest key, so all inserts land on the shard owning the top range until the balancer moves it, creating a permanent hot shard.
Sources and further study
Primary
- Jepsen: MongoDB 3.6.4: which causal-session guarantees survive partitions, and under which concerns.
- MongoDB 8.0 release notes: moveable and unsharded collections, config shards and faster resharding.
Systems docs
- MongoDB: Write concern: the implicit default, the arbiter exception and journaling.
- MongoDB: Replica set oplog: default sizing and minimum retention.
Books
- Designing Data-Intensive Applications (1st edition), Martin Kleppmann, chapter 2 "Data Models and Query Languages" and chapter 6 "Partitioning": the document model's locality trade-off and key-range versus hash partitioning.
- Database Internals, Alex Petrov, chapter 14 "Consensus": Raft, which MongoDB's elections follow.
Other curricula
- donnemartin/system-design-primer: lists MongoDB as a document store with a query language, contrasted with key-value and wide-column stores.
- roadmap.sh: MongoDB: a step-by-step learning path for MongoDB developers, useful as a checklist beyond the storage internals covered here.
- Azure Architecture Center: Understand data models: frames document stores as denormalised aggregates with per-document atomicity and warns about document growth.
Watch
- How MongoDB replication follows and improves upon the Raft algorithm (Henrik Ingo, PG Day Russia): how pull-based oplog replication maps onto Raft.
- Keith Bostic: WiredTiger (CMU Database Group): the storage engine's design from one of its authors.
- Fixing a MongoDB Replication Protocol Bug with TLA+ (William Schultz): where the protocol diverges from Raft and how a model checker found a bug.
Free resources
- Tunable Consistency in MongoDB (VLDB 2019): MongoDB engineers on write and read concern and their measured costs.
- Replica set elections (MongoDB docs): heartbeats, timeouts, priorities and what triggers an election.
- WiredTiger storage engine (MongoDB docs): cache sizing, compression, checkpoints and journaling.
- Sharded cluster balancer (MongoDB docs): migration thresholds and the range migration procedure.
- Jepsen: MongoDB 4.2.6: what transactions and weaker concerns actually guaranteed under faults.
- In Search of an Understandable Consensus Algorithm (Raft paper): the protocol MongoDB's replication follows.