Engineering Atlas › System Design Interviews

Design an email service

20 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

Requirements

Functional

Non-functional

Out of scope: chat, Meet, calendar invites beyond plain attachments, the ads system, and the internals of the spam model.

Back-of-envelope

Assumed round numbers, not Google's figures.

API

SMTP (port 25, from other mail servers)
  EHLO, STARTTLS, MAIL FROM:<a@x.com>, RCPT TO:<b@gmail.com>, DATA ... "."
  -> 250 OK (durably accepted) | 4xx (try later) | 5xx (permanent reject)

POST /v1/messages/send           { raw_mime_or_fields, client_request_id }
                              -> { message_id, thread_id }
GET  /v1/threads?label=INBOX&page_token=..&limit=50
                              -> { threads: [ { thread_id, snippet, last_date, unread, senders } ], next_page_token }
GET  /v1/threads/{thread_id}  -> { messages: [ { message_id, headers, body_ref, attachments } ] }
POST /v1/messages/{id}/modify { add_labels: ["STARRED"], remove_labels: ["UNREAD"] }
GET  /v1/search?q=from:alice has:attachment after:2026/01/01&page_token=..
GET  /v1/history?start_history_id=918273&limit=500
                              -> { changes: [ { history_id, message_id, kind, labels } ], history_id: 918301 }
IMAP4rev2 (port 993) for third-party clients, mapped onto labels and the history log

send is idempotent on client_request_id, so a retried tap on a phone does not send twice. The history endpoint mirrors the public Gmail API, where history.list takes a startHistoryId and returns everything that changed after it.

Data model

messages partitioned by user_id, primary key (user_id, message_id). Columns: thread_id, internal_date, labels (a small set), size, snippet, header_digest, body_ref, rfc822_message_id. A user's mailbox lives on one shard, so a label change and its counter update are one local transaction.

label_index primary key (user_id, label, internal_date DESC, message_id). This is the inbox: the first page of INBOX is a prefix scan returning 50 rows. Every label add or remove writes or deletes one index row.

threads primary key (user_id, thread_id). Columns: subject_norm, last_date, message_count, label_union. Thread lists are served from a parallel (user_id, label, last_date DESC, thread_id) index so a conversation appears once, at the time of its newest message.

label_counts (user_id, label) -> total, unread. Updated in the same transaction as the label change; a background job recounts periodically to repair drift.

history primary key (user_id, history_id). One row per change (message added, deleted, labels changed). history_id is a per-user monotonic counter, and old rows are trimmed. Gmail promises history for typically at least a week, sometimes less, and answers a too-old startHistoryId with HTTP 404, which tells the client to do a full sync (Gmail API).

bodies and attachments live in a blob store keyed by content hash. The same 5 MB PDF sent to 2,000 employees is stored once with a reference count, and the body store is encrypted at rest with per-user or per-tenant keys.

search index is a per-user inverted index, also on the user's shard: terms to posting lists of message_ids, plus fields for from, to, subject and filename.

High-level design

flowchart LR
  EXT["Sending mail servers"] -->|"SMTP"| MX["MX edge: TLS, rate limits"]
  MX --> AUTH["SPF / DKIM / DMARC + reputation"]
  AUTH --> SPAM["Spam and malware classifier"]
  SPAM --> DEL["Delivery service: filters, threading"]
  DEL --> META[("Mailbox shards by user_id")]
  DEL --> BLOB[("Body and attachment blob store")]
  DEL --> IDX["Per-user search indexer"]
  DEL --> PUSH["Notification service"]
  CL["Web / mobile / IMAP clients"] --> API["Mail API"]
  API --> META
  API --> BLOB
  API --> OUTQ["Outbound queue"]
  OUTQ --> MTA["Outbound MTAs: DKIM sign, retry"]
  MTA -->|"SMTP"| EXTR["Recipient mail servers"]

Inbound write path.

  1. A sending server looks up the MX record for gmail.com and opens an SMTP connection to the nearest edge. The edge checks connection rate limits and IP reputation, and can refuse before reading a byte of content.
  2. During the SMTP dialogue the edge verifies SPF on the envelope sender, then reads the message, verifies DKIM signatures and evaluates the DMARC policy of the visible From domain.
  3. The classifier scores the message. Clear spam can still be accepted and filed under the Spam label; clear malware is rejected with a 5xx.
  4. The delivery service writes the body to the blob store, then, in one transaction on the recipient's shard, inserts the message row, label index rows, counter updates and a history row. It runs the user's filters and the threading logic first, so the labels and thread_id are known at insert.
  5. Only after that transaction commits on a quorum of replicas does the edge reply 250 OK. For a message to many local recipients, each recipient's delivery is a separate idempotent transaction keyed by (user_id, rfc822_message_id, queue_id).
  6. The indexer adds the message to the user's search index asynchronously, and the notification service wakes connected clients.

Read path. The client asks for the INBOX thread list: a prefix scan of the thread-label index on the user's shard, with snippets stored inline so no bodies are fetched. Opening a thread reads its message rows and fetches bodies from the blob store (cached), with attachments downloaded on demand through signed URLs. Background clients poll or are pushed a new history_id and call history to apply only the changes.

Outbound path. send writes the message to the sender's Sent label and enqueues it. Outbound MTAs sign with DKIM, group by destination domain, and deliver with per-domain concurrency limits. A 4xx response schedules a retry with backoff for up to several days; a 5xx produces a bounce message back to the sender.

Deep dives

1. Accepting mail without losing it or drowning in spam

The SMTP contract is the heart of the system: after 250, the sending server forgets the message. Options: accept into a local disk queue on the edge and deliver later (classic Postfix design, fast but the message lives on one machine for a while), or deliver synchronously into replicated storage before answering (slower SMTP sessions, no single-machine window). At this scale I would take a middle path: the edge writes the message to a replicated inbound queue (a quorum across three zones) before answering, and delivery workers drain the queue into mailboxes. The 250 then means "durable in three zones", and a dead edge loses nothing.

Spam defence is layered by cost. Connection-level checks (IP reputation, rate limits, DNS blocklists) are microseconds and stop most abuse before DATA. Authentication comes next: SPF checks that the connecting IP may send for the envelope domain, DKIM verifies a signature over headers and body with a key published in DNS (RFC 6376), and DMARC requires that one of them aligns with the visible From domain and tells receivers what to do when neither does. DMARC was an Informational RFC (7489) for a decade; in May 2026 it became Standards Track as RFC 9989, which replaces the public suffix list with a bounded DNS tree walk to find the organizational domain and drops the pct tag (RFC 9989). Records still start with v=DMARC1. Since February 2024 Gmail requires every sender to use SPF or DKIM, TLS and a spam rate under 0.3%, and senders of more than 5,000 messages a day to Gmail to also pass DMARC alignment and support one-click unsubscribe (Google sender guidelines), using the List-Unsubscribe-Post header from RFC 8058. Only messages that survive reach the expensive classifier, which uses content, sender history and the recipient's own behaviour. This is the overload problem shape solved by ordering filters from cheapest to most expensive, the same structure a web application firewall or an API gateway uses.

2. Labels, threads and counters on one shard

Folders would mean storing a message in one place and moving it; labels let a message be in Inbox and Work at once without copies. The cost is an index row per label per message and a counter per label, all of which must agree. Keeping one user's data on one shard makes every mailbox mutation a single-shard transaction, which avoids distributed transactions entirely. The penalty is that a giant mailbox (a shared support inbox with 50M messages) makes a big shard; split by user into virtual shards and move heavy users individually.

Threading has no perfect answer. Headers say who replied to whom: In-Reply-To and References carry the RFC 5322 Message-IDs of earlier messages. Subjects say what humans think a conversation is. Gmail combines them: its API adds a message to a thread only when the References and In-Reply-To headers point into it and the subject matches (Gmail API), so changing the subject starts a new conversation, and two unrelated "Hello" emails never merge on subject alone. Messages also arrive out of order (a reply can land before its parent), so the threader must remember which unknown ids have been referenced and merge threads when the missing link appears. The practice task builds exactly this.

3. Search per user

A global inverted index over 5 EB of mail would be enormous, and every query would have to filter by owner, a security risk if the filter is ever wrong. A per-user index is small (a typical mailbox of 20,000 messages indexes into tens of megabytes), lives next to the mailbox, and isolates users by construction. Indexing happens asynchronously on delivery, so a message may take a few seconds to become searchable; that is an acceptable relaxation, while the inbox itself must be immediate.

The challenge is the long tail. Most users search recent mail, so recent segments sit on SSD or in memory and old segments on cheaper storage, merged LSM-style. Very large mailboxes get their index split by time range and queried in parallel. Operators like from:, has:attachment and after: map to fielded postings and a date-sorted docid order, so a query for recent mail can stop as soon as it has enough hits.

Down the stack

Layer 1The contract

  • Inbound. A 250 reply to the end of DATA means the message is durable on at least three replicas in different zones and will appear in every listed recipient's mailbox exactly once. A 4xx means nothing was kept and the sender should retry; a 5xx means a permanent refusal.
  • Mailbox. A label change, its index rows, its counters and its history row commit atomically per user. A read after a write by the same client sees the write.
  • History. history_id increases with every change in a mailbox. Reading after N returns every change with id greater than N in order, or tells the client its cursor is too old.
  • Outbound. An accepted send is delivered, bounced with a reason, or reported as delayed; it is never silently dropped and never sent twice for the same client_request_id.

Layer 2The mechanism

SMTP (RFC 5321) is a line-based command and reply protocol, and its successor, rfc5321bis, is approved and waiting in the RFC Editor queue as an Internet Standard (IETF datatracker): EHLO advertises extensions such as STARTTLS, SIZE, PIPELINING and 8BITMIME; MAIL FROM sets the envelope sender used for bounces; one RCPT TO per recipient; DATA streams the message until a line containing a single dot. The message itself (RFC 5322 headers plus MIME parts) is separate from the envelope, which is why SPF checks the envelope and DMARC checks the header From.

A DKIM signature is a header with tags such as d= (signing domain), s= (selector), bh= (hash of the canonicalised body) and b= (signature over the chosen headers). The receiver fetches the public key from <selector>._domainkey.<domain> in DNS. MTA-STS (RFC 8461) lets a domain require TLS for inbound delivery so a network attacker cannot strip STARTTLS.

IMAP, now at IMAP4rev2 (RFC 9051), gives every message a UID that only increases within a mailbox, plus a UIDVALIDITY value that changes if UIDs are ever reassigned, which forces clients to resync. The CONDSTORE and QRESYNC extensions (RFC 7162) add a per-message modification sequence, the IMAP equivalent of history_id. Gmail exposes labels to IMAP clients as folders, so a message with two labels appears in two IMAP folders.

Layer 3The machine

  • Bytes on the wire. A 50 KB message is about 35 TCP segments at a 1,448-byte MSS. Base64 turns every 3 bytes of attachment into 4 characters, so a 15 MB attachment travels as 20 MB, which is why the 25 MB attachment limit sits well below the larger total message size limit.
  • TLS cost. A full TLS 1.3 handshake with an ECDHE key exchange costs about a millisecond of CPU on the server. Millions of short SMTP connections per second make that a real fleet cost, so session resumption and connection reuse by large senders matter.
  • Durable accept. Waiting for a quorum write across zones adds 1 to 3 ms of network round trips plus a log fsync on each replica (tens of microseconds on NVMe with power-loss protection, milliseconds on spinning disks). SMTP sessions last hundreds of milliseconds anyway, so this is affordable.
  • Index layout. Posting lists store sorted docids as delta-encoded varints, often a byte or two per entry. A 20,000-message mailbox with 300 distinct terms per message is 6M postings, roughly 10 MB, which reads from SSD in a few milliseconds.
  • Dedup by hash. SHA-256 over a 5 MB attachment takes about 3 ms; one hash lookup then saves storing it 2,000 times.

Why it's built this way

  1. SMTP senders delete their copy after 250, so the receiving side must make the message durable before saying yes.
  2. Spam outnumbers real mail, so filters run in order of cost and reject as early in the SMTP dialogue as possible.
  3. Mail is private and accessed per user, so partitioning by user makes every mailbox change a local transaction and every search a single-shard query.
  4. Users want one message in several views, so labels plus per-label indexes replace folders, and counters update in the same transaction.
  5. Clients go offline and many of them are third-party, so sync is a per-user ordered change log, exposed as history ids to Google's own apps and as UIDs and modification sequences to IMAP.

Failure modes

In the wild

Trade-offs

OptionStrengthsWeaknessesChoose when
Edge disk queue before deliveryFast SMTP replies, simpleMessage on one machine until deliveredSmall or single-site mail systems
Replicated queue before 250No single-machine loss windowExtra latency and infrastructureLarge providers
LabelsOne copy, many viewsIndex row and counter per labelMulti-view mailboxes like Gmail
FoldersSimple model, maps to IMAPOne location per messageClassic mail servers
Per-user search indexIsolation, small and localMany small indexes to managePrivate data per user
Global search indexCross-user queries, fewer filesHuge, every query must filter by ownerShared corpora, not personal mail
Thread by headers and subjectMatches user expectationEdge cases with edited subjectsConsumer mail

Leadership lens

Check yourself

  1. Why does the edge write to a replicated inbound queue before replying 250 OK, instead of queuing on its local disk?

    Show answer

    After 250 the sending server deletes its copy, so the message must already survive any single machine failure. A local disk queue is fast but leaves the message on one machine until delivery, while a quorum write across three zones costs only a few milliseconds inside an SMTP session that lasts hundreds of milliseconds anyway.

  2. A spam wave or huge mailing brings 10x inbound load. How should the system respond without losing legitimate mail?

    Show answer

    Shed at the edge first by tightening per-IP and per-sender-domain rate limits and answering 4xx, so legitimate senders simply retry later. The queue in front of delivery workers smooths the rate that mailbox shards actually see.

  3. Why use a per-user search index rather than one global inverted index?

    Show answer

    A global index over exabytes of mail would be enormous and every query would have to filter by owner, which is a security risk if that filter is ever wrong. A per-user index is small, lives on the user's shard and isolates users by construction, at the cost of managing many small indexes.

  4. Roughly how much storage do message bodies need compared with metadata, and what does that imply for cost?

    Show answer

    About 60B messages a day at 50 KB each is 3 PB a day, around 5.5 EB over five years, while 1 KB metadata rows add only about 60 TB a day. Bodies and attachments dominate cost, so compression, attachment dedup by content hash and cold-tier storage pay the most.

Sources and further study

Primary

Systems docs

Books

Other curricula

Watch

Free resources

Open in the interactive app: practice, quiz, mock interview →