Engineering Atlas › System Design Interviews

Elasticsearch for system design interviews

13 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

How it works

Documents, mappings and analysis

A document is JSON in an index. The mapping says how each field is indexed: text fields pass through an analyzer (tokenizer, lowercasing, stemming, synonyms) into terms; keyword fields are exact values; numbers, dates and geo points go into trees for range queries. Changing an existing field's type means reindexing.

Shards and routing

A document goes to shard hash(_routing) % number_of_primary_shards, with _routing defaulting to the ID. The primary count (default 1) is fixed at creation (Elastic docs), so time-based data uses one index per day or size threshold, managed by data streams and lifecycle policies that roll over and delete old indexes.

Search

A coordinating node sends the query to one copy of every relevant shard. Each shard returns its local top hits; the coordinator merges them, then fetches full documents for the final page. Aggregations compute partial results per shard over columnar doc values and reduce them centrally.

Down the stack

Layer 1The contract

An index request returns success once the operation is in the primary's translog and has been replicated to all in-sync replicas. By default the translog is fsynced on the primary and every allocated replica before acknowledging (index.translog.durability: request), so an acknowledged write survives a crash (Elastic docs). The alternative, async, fsyncs every sync_interval (5 s) and can lose that much; Elasticsearch 1.x used that behaviour by default, which is part of why Jepsen saw acknowledged writes vanish. A write is not visible to search until the next refresh, up to index.refresh_interval (1 s, or 5 s on Serverless) later; a GET by ID, however, is real-time. One catch: if you never set refresh_interval explicitly, a shard that has seen no search for index.search.idle.after (30 s) skips background refreshes, and the first search after that waits for one (Elastic docs). Concurrency control is optimistic: each document carries _seq_no and _primary_term, and a write with if_seq_no/if_primary_term fails with a conflict if someone else changed it. There are no multi-document transactions. Searches are not snapshot-consistent across shards unless you use a point-in-time. Typical latency: single-digit ms for an indexed write (bulk requests amortise this), 10 to 100 ms for a search, depending on shard count and query cost.

Layer 2The mechanism

Lucene segments. A shard is a Lucene index made of segments. Each segment is a self-contained, immutable mini-index with its own files: a term dictionary stored as a finite state transducer (.tip) pointing into blocks of terms (.tim); postings lists of document IDs and term frequencies (.doc), positions (.pos) and offsets or payloads (.pay); stored fields holding _source, compressed in blocks (.fdt); doc values, a column store for sorting and aggregations (.dvd); norms holding each document's field length for scoring (.nvd); BKD trees for numeric and geo points (.kdd, .kdi); and a live-docs bitset (.liv) marking deleted documents. Postings are delta-encoded and bit-packed in blocks of 128 documents, still the fixed block size in Lucene 10 (Lucene docs), and skip data stored with them lets a query jump over whole blocks when intersecting terms.

Indexing. New documents are analysed and added to an in-memory buffer and appended to the translog. A refresh writes the buffer out as a new segment and opens it for search. Updates are a delete (set a bit in .liv) plus an insert of the new version. A flush performs a Lucene commit, fsyncing segments and starting a new translog generation. A background merge policy (tiered, targeting segments up to about 5 GB) combines small segments into larger ones and physically drops deleted documents.

Replication. Elasticsearch uses primary-backup replication. The master tracks an in-sync set of shard copies by allocation ID. The primary assigns each operation a sequence number, applies it, forwards it to every in-sync replica in parallel, and acknowledges when all respond. A replica that fails to respond is reported to the master and removed from the in-sync set before the primary acknowledges, so an acknowledged write is on every copy that might be promoted (Elastic docs). The global checkpoint, the highest sequence number all in-sync copies have processed, lets a returning replica replay only the operations it missed from the translog or retained history, instead of copying whole segments.

Cluster coordination. Master-eligible nodes elect a master by quorum: a decision needs responses from more than half of the voting configuration, so stopping half the master-eligible nodes at once makes the cluster unavailable (Elastic docs). The master owns cluster state (mappings, allocation, in-sync sets) and publishes changes in two phases, committing after a quorum acknowledges.

Search execution. In the query phase each shard evaluates the query against each segment, combining postings iterators (conjunctions leapfrog via skip lists), scoring with BM25 using the norms (defaults k1 = 1.2, b = 0.75; Elastic docs), and keeping a priority queue of from + size hits. Block-max WAND skips blocks whose maximum possible score cannot reach the current top-k threshold, which makes top-10 queries over common terms fast. The coordinator merges per-shard queues, then the fetch phase loads _source for the winners.

Layer 3The machine

Segments are written once and read through the OS page cache, often memory-mapped with mmap. Search speed depends on how much of the hot index fits in that cache, which is why the heap should stay at or below half of RAM and below the compressed-pointer threshold, which Elastic now puts at 26 GB as safe and up to 30 GB on some systems, leaving the rest to the kernel. Modern versions size the heap automatically from the node's roles and memory, and Elastic recommends keeping that default (Elastic docs). A query whose postings are cached costs microseconds per segment; one that misses goes to SSD at about 100 us per random read, or to spinning disk at about 10 ms, which is why spinning disks are a poor fit for search tiers.

Each refresh creates a small segment and future merge work; bulk loads with refresh_interval: -1 and zero replicas, restored afterwards, can ingest several times faster. Each node has fixed-size thread pools: write sized to the allocated processors and search to int(processors * 3 / 2) + 1, each with a bounded queue. The queue sizes recently changed: search now queues 1,000 per thread (a flat 1,000 in 8.x and earlier) and write queues max(10000, processors * 750) (a flat 10,000 through 9.1) (Elastic docs). When the queue fills, requests are rejected with HTTP 429, which clients must treat as back-pressure. Heap pressure comes from aggregations over high-cardinality fields, large size values and deep pagination; circuit breakers abort requests that would exceed configured heap fractions instead of letting the node run out of memory.

Why it's built this way

  1. Search needs to find documents by term, the inverse of how they are stored, so the core structure is an inverted index from term to sorted document IDs.
  2. Updating inverted indexes in place is expensive random I/O, so Lucene writes immutable segments and merges them later, like an LSM tree.
  3. Writing a segment for every document would be too slow and too many files, so documents buffer in memory and refresh periodically, which makes search near-real-time rather than real-time.
  4. Buffered documents would be lost in a crash, so a translog provides durability independently of segment commits, which are expensive fsyncs done rarely.
  5. One machine cannot hold or scan billions of documents quickly, so indexes are sharded and replicated, and queries scatter and gather.

Diagram

flowchart LR
    C["Client bulk request"] --> CO["Coordinating node"]
    CO -->|"hash(_routing) % shards"| P["Primary shard"]
    P --> BUF["In-memory buffer"]
    P --> TL["Translog (fsync)"]
    P --> R1["Replica (in-sync)"]
    R1 -->|"ack"| P
    P -->|"ack"| CO
    BUF -->|"refresh every 1 s"| SEG["New searchable segment"]
    SEG -->|"flush: Lucene commit + fsync"| DISK["Committed segments"]
    DISK -->|"tiered merge"| BIG["Larger segments, deletes purged"]

Worked example

An e-commerce site has 50 million products of about 2 KB, 3,000 searches per second at peak and 500 price or stock updates per second.

Raw data is 100 GB; with analysed text, doc values and _source, call it 130 GB per copy, so 4 primaries of about 32 GB fit the 10 to 50 GB guidance. If a query costs about 20 ms of CPU per shard, demand is 3,000 x 4 x 20 ms = 240 CPU-seconds per second. Two replicas (3 copies) on 12 data nodes of 32 cores give 384 cores, about 60 percent busy at peak, and each node holds one 32 GB shard copy that fits the page cache of a 64 GB node with a 26 GB heap.

Each update rewrites the whole document and marks the old version deleted, so merges reclaim space continuously. Results lag updates by up to a second, so "add to cart" reads price and stock from the transactional database, never from the search hit.

In the wild

Trade-offs

OptionStrengthsWeaknessesChoose when
Elasticsearch or OpenSearchFull-text relevance, aggregations, mature ecosystemHeap tuning, shard sizing, not a primary storeProduct search, log analytics, faceted navigation
Postgres full-text searchOne system, transactional, simpleWeaker relevance tuning, harder to scale past one nodeModest search needs alongside relational data
SolrLucene-based, strong faceting, open governanceSmaller ecosystem todayTeams already invested in Solr
ClickHouse or a column storeVery fast aggregations, cheap storageWeak full-text relevanceLog and event analytics without relevance ranking

Pitfalls in production

Leadership lens

Check yourself

  1. A user updates a product and immediately searches for it, but sees the old version. Why, and what are the options?

    Show answer

    The update is in the translog and buffer but not yet refreshed into a searchable segment; read by ID (real-time), wait for the next refresh with refresh=wait_for, or accept up to a second of lag.

  2. Why are deletes in Elasticsearch cheap at write time but costly later?

    Show answer

    A delete only sets a bit in the segment's live-docs file; the space and the cost of skipping the document remain until a merge rewrites the segment.

  3. What does BM25's b parameter control?

    Show answer

    How strongly scores are normalised by field length; b = 0 ignores length, b = 1 fully normalises, and the default 0.75 penalises long fields moderately.

  4. Why keep the JVM heap under about 31 GB?

    Show answer

    Above that the JVM loses compressed object pointers, so a larger heap holds fewer objects, and memory taken from the page cache slows segment reads.

Sources and further study

Primary

Systems docs

Books

Other curricula

Watch

Free resources

Open in the interactive app: practice, quiz, mock interview →