Design a Google Docs-style collaborative editor
Open in the interactive app: practice, quiz, mock interview →In one minute
- Pick the concurrency model first. A central server that assigns every operation a revision number and transforms stale operations (client-server operational transformation) is simpler to reason about than peer-to-peer schemes and fits an always-online editor. CRDTs are the alternative when offline and peer-to-peer matter more.
- Give each document exactly one live owner process at a time. That owner holds the document in memory, orders edits, and is the only writer of the document's operation log. Fence it so a deposed owner cannot write.
- Ack an edit only after it is durably appended to the operation log at its revision. The log, plus periodic snapshots, is the document.
- The client edits optimistically against its local copy, keeps one operation in flight, and buffers the rest, so typing never waits on the network.
- Presence (cursors, selections) is ephemeral and best-effort; it should never share a durability path with edits.
Requirements
Functional:
- Create, open and edit rich-text documents; several people edit at once and see each other's changes within about a second.
- See collaborators' cursors and selections.
- Comments anchored to text ranges, version history with named versions, and restore.
- Share with view, comment or edit permissions.
Non-functional:
- Convergence: everyone who has received the same set of edits sees the same document.
- Intent preservation: concurrent edits are merged in a way that matches what each author meant (typing in different places should both survive).
- Local keystroke latency under 50 ms regardless of network; remote edits visible in under 500 ms p95 within a region.
- No acknowledged edit is ever lost. High availability for opening and reading documents.
Out of scope: full offline editing for days (we tolerate short disconnects), spreadsheets and slides, spelling and grammar services, and export formats.
Back-of-envelope
Planning assumptions, not published figures.
- DAU: 50M people who edit at least once. At the busiest hour about 5M are actively editing at the same moment.
- Clients batch keystrokes and send at most one operation batch every 100 to 200 ms while typing, and nothing while idle. Averaged over active editors (who pause to read and think), say 0.3 batches per second each. Peak write rate: 5M × 0.3 = 1.5M operations per second. Average across the day is perhaps a fifth of that, around 300,000 per second.
- Fan-out: most documents have one editor; a few have dozens. If the average active document has 1.5 connected viewers, broadcast traffic is about 1.5 × 1.5M ≈ 2.25M messages per second at peak.
- Operation size: about 100 bytes with metadata. Daily op volume: 300,000/s × 86,400 s × 100 B ≈ 2.6 TB per day of raw log. Over 5 years that would be about 4.7 PB, which is why logs are compacted into snapshots and only coarse version history is kept forever.
- Snapshots: say 2B documents averaging 50 KB of content and formatting, about 100 TB, replicated three ways to about 300 TB.
- Bandwidth: 2.25M messages × 100 B ≈ 225 MB/s out at peak, plus about 150 MB/s in. Document opens dominate bytes: 20M opens per hour × 50 KB ≈ 280 MB/s.
- Connections: 5M concurrent WebSockets at roughly 20 to 50 KB of kernel and app memory each is 100 to 250 GB of RAM spread over the gateway fleet, so a few hundred gateway machines.
API
GET /v1/docs/{docId} -> {snapshot, rev, acl}
WS /v1/docs/{docId}/session (after auth and ACL check)
client -> server
{type:"op", clientId, seq: 41, baseRev: 1207, op:[{retain:15},{insert:"hi"}]}
{type:"presence", cursor: 17, selection:[17,22]}
server -> client
{type:"ack", seq: 41, rev: 1209}
{type:"op", rev: 1208, author, op:[...]} // others' ops, in rev order
{type:"presence", user, cursor, selection}
{type:"resync", fromRev: 1180} // client fell too far behind
GET /v1/docs/{docId}/ops?fromRev=1180&toRev=1209
GET /v1/docs/{docId}/versions -> [{rev, time, authors, name?}]
POST /v1/docs/{docId}/comments {anchorRev, start, end, body}clientId plus seq makes resends after a reconnect idempotent: the server remembers the highest seq per client and returns the original ack for a duplicate.
Data model
documents:doc_id PK, owner_id, title, latest_snapshot_rev, latest_rev, created_at. Partition bydoc_id.doc_ops:(doc_id, rev) PK, client_id, seq, author_id, op_bytes, ts. Partition bydoc_id, clustered byrev, so loading "everything after the snapshot" is one contiguous range scan. A conditional insert that fails if(doc_id, rev)already exists is the core write; it doubles as a fencing check.doc_snapshots:(doc_id, rev) PK, content_blob_ref. Written every N operations (say 500) or every few minutes of activity; the blob lives in object storage. Old ops behind a snapshot are kept for 30 days for fine-grained history, then compacted.acl:(doc_id, principal) PK, role, with a reverse index(principal, doc_id)for "shared with me". Cached at the gateway with short TTLs; revocation pushes a disconnect to live sessions.comments:(doc_id, comment_id) PK, anchor_start, anchor_end, anchor_rev, thread. Anchors are transformed through later ops the same way cursors are.session_owner(in a coordination service):doc_id -> {server, epoch}, held under a lease.
A store with per-row conditional writes and strong per-partition ordering fits doc_ops: Bigtable or Spanner-like systems, Cassandra with lightweight transactions at a cost, or sharded Postgres. Snapshots go to object storage because they are large, immutable and read in full.
High-level design
flowchart LR
C["Editor client"] -- WebSocket --> GW["Gateway"]
GW --> RT["Doc router"]
RT --> DS["Doc session owner"]
CO["Coordination: owner leases"] --> DS
DS -- "append (doc, rev)" --> LOG[("Op log")]
DS --> SNAP[("Snapshots in blob store")]
DS -- "broadcast ops" --> GW
GW -- "presence" --> PR["Presence pub/sub"]
C -- HTTPS --> API["Doc API"]
API --> META[("Docs, ACL, comments")]
API --> SNAPWrite path. A user types. The client applies the edit to its local model immediately, then, if no operation is in flight, sends it with the revision it was based on; otherwise it composes it into a buffer. The gateway forwards the message to the document's session owner, found through the router's lookup of session_owner. The owner checks for a duplicate (clientId, seq), then transforms the operation against every operation in its in-memory log after baseRev, assigns the next revision, and does a conditional append of (doc_id, rev) to the op log. When the append succeeds it applies the op to its in-memory document, acks the author with the revision, and broadcasts the transformed op to every other connected client. Each recipient transforms it against its own in-flight and buffered operations before applying it.
Read path. Opening a document fetches the latest snapshot and the ops after it (or asks the session owner, which has both in memory), renders, and opens the WebSocket with that revision as its starting point. Viewers without edit permission get the same op stream, read-only. Version history reads snapshots and replays ops up to the chosen revision.
Deep dives
Ordering concurrent edits: OT or CRDT
Options considered. First, locking paragraphs. It is simple but feels broken: people block each other. A close cousin is diffing whole document versions and merging them on the server, which is what Google Docs did before 2010; it mis-merges when one person restyles words another is replacing (Google Drive blog). Second, CRDTs (as in Yjs and Automerge): every character gets a unique, ordered identity, and merges commute, so no central ordering is needed. They are excellent for offline and peer-to-peer use, at the cost of per-character metadata (much reduced by modern implementations: one 2021 rewrite replayed an editing trace in 56 ms that took an older Automerge nearly 5 minutes, Joseph Gentle) and tombstones that must be garbage collected. Convergence alone is not enough: several published text CRDTs can interleave two people's concurrently typed words character by character (Kleppmann et al., PaPoC 2019), so check the algorithm, not just the library's popularity. Third, client-server OT in the style of the Jupiter system, which is widely reported to underlie Docs: the server keeps a single linear history, each client has at most one unacknowledged operation, and the server transforms a stale operation against the operations it missed. Google described exactly this protocol in 2010: the document is a revision log of insert-text, delete-text and apply-style changes, and a client "never" sends more than one pending change at a time (Google Drive blog, part 3).
Choice: client-server OT. With one ordering point, we only need the transform function to satisfy the simpler convergence property for two concurrent operations (often called TP1); the harder property needed for peer-to-peer OT (TP2) is avoided because the server serialises everything (Wikipedia: Operational transformation). The research line has kept moving: Eg-walker (2024) stores an OT-like event graph but merges like a CRDT, using an order of magnitude less steady-state memory than existing CRDTs and merging long offline branches orders of magnitude faster than OT (Gentle and Kleppmann). It is worth naming if the interviewer pushes on offline editing. The client keeps three states: synchronised, awaiting ack, and awaiting ack with a buffer. That bounds the transform work to "ops the server applied since my base revision". Rich text adds formatting attributes, but the structure is the same: operations are sequences of retain, insert and delete with attributes, and the transform walks two sequences in step.
One owner per document
OT needs a single serialisation point per document, so each document is assigned to one session server by consistent hashing over the doc ID, confirmed by a lease in the coordination service that carries an increasing epoch. The owner keeps the current text, the recent op log (enough to transform any reasonably stale op) and the connected client list in memory. If a client's baseRev is older than the in-memory window, the server asks it to resync.
Split brain is the danger: an owner pauses, its lease expires, a new owner takes over, and the old one wakes and keeps accepting edits. The conditional append to doc_ops is what makes this safe. Both owners compute "next rev = 1209", only one append can create (doc_id, 1209), and the loser discovers it is stale, drops its sessions and lets clients reconnect to the new owner. Some stores also let us check the epoch inside the same conditional write, which rejects the stale owner even earlier.
Storage, snapshots and history
Replaying a million ops to open a long-lived document would be slow, so the owner writes a snapshot every 500 ops or so, asynchronously, to the blob store and records its revision. Opening a document then costs one blob read plus at most a few hundred op reads. History is kept at three resolutions: every op for 30 days (powers "see who typed this" and fine-grained restore), periodic snapshots grouped into sessions for the visible history list, and named versions kept forever. Compaction merges old ops into the snapshots and deletes the log rows. Comments and cursors are positions in a moving document, so they are stored with the revision they were anchored at and transformed forward through later ops, the same way cursors are.
Down the stack
Layer 1The contract
Every acknowledged operation has a unique revision number, and revision numbers for a document are dense and increasing. All clients apply the same operations in revision order (after their own local transforms), so any two clients at the same revision show identical documents. An ack means the operation is durable in the op log. A resend with the same (clientId, seq) is acknowledged once and applied once. Presence messages have no guarantees: they may be dropped or reordered and are superseded by the next update.
Layer 2The mechanism
The server-side transform for one incoming op A with base revision b is a fold: for each logged op B with revision greater than b, replace A with transform(A, B). For inserts at the same position the rule must be deterministic; we let the already-logged insert win, so the incoming text lands after it. A delete of a character that a concurrent op already deleted becomes a no-op. The client runs the mirror image: when a remote op R arrives, it transforms R against its in-flight op P and buffer Q, applies the transformed R, and replaces P and Q with their transformed versions, so its own pending edits stay valid against the new base. When the ack arrives, the buffer becomes the in-flight op and is sent with the acked revision as its base. Durability is a conditional put on (doc_id, rev); in Bigtable-style stores that is a check-and-mutate on the row, and in Postgres it is an INSERT that fails on the primary key.
Layer 3The machine
A keystroke batch serialises to roughly 60 to 150 bytes of JSON or a few dozen bytes of a binary encoding, sent as one WebSocket frame inside one TCP segment. Gateways set TCP_NODELAY so Nagle's algorithm does not hold small frames for up to 40 ms waiting for an ACK. A gateway multiplexes tens of thousands of sockets with epoll_wait; each idle connection costs a few kilobytes of kernel socket buffer plus application state. Round trips dominate: 10 to 40 ms within a region, 100 to 200 ms across continents, which is why session owners live in the region nearest most editors and clients never wait for acks to show their own typing. The durable append is the slowest server step: a replicated write across zones costs a few milliseconds, and an owner pipelines appends so throughput is not limited to one op per round trip. A 50 KB document plus a 1,000-op tail is well under 1 MB of heap, so one server can own tens of thousands of active documents.
Why it's built this way
- Network round trips are tens to hundreds of milliseconds and typing must feel instant, so clients apply edits locally and reconcile later.
- Local-first editing makes concurrent edits inevitable, so we need a merge rule; one central ordering point keeps that rule simple enough to get right.
- A single orderer per document is a single point of failure, so ownership is a lease that moves, with conditional appends to fence old owners.
- An ever-growing log makes opens slow and storage expensive, so we snapshot and compact, keeping fine detail only where users actually need it.
Failure modes
- 10x load. Gateways scale horizontally behind the load balancer. Session owners scale by document count; the router rebalances hash ranges as servers are added, migrating documents by draining their sessions. If the op log store is the bottleneck, owners batch several ops per append.
- Session owner crash. Clients see the socket close, back off with jitter, and reconnect. The new owner loads the latest snapshot and log tail, clients resend their in-flight op with the same
(clientId, seq), and dedupe prevents double application. Users see a "reconnecting" banner for a few seconds; typing continues locally throughout. - Region outage. The op log is replicated synchronously across zones and asynchronously across regions. Documents fail over to an owner in another region; with asynchronous replication, the last second of acked ops in the failed region could be missing. Clients still hold those ops as acked but not reflected, so the reconnect protocol compares revisions and resubmits from the client's copy where possible.
- Hot document. A company all-hands doc with 500 editors and 50,000 viewers. Editors stay on the owner; viewers are served by a read-only fan-out tier that subscribes to the owner's stream, so broadcast cost leaves the critical path. Cap simultaneous editing sessions and degrade the rest to view or comment. Google's own limit is that a file can be edited from at most 100 open tabs or devices at once; beyond that only the owner and some editors can edit (Google Docs Editors Help).
- Slow or malicious client. A client that never acks or floods ops gets rate-limited per connection, and its outbound queue is bounded; beyond the bound it is told to resync.
In the wild
- Google Docs, 2010: the rebuilt editor is client-server OT. Google replaced the old diff-and-merge Docs with a revision log of three change types (insert text, delete text, apply style) that is replayed to display a document. Clients apply edits locally, send at most one pending change, and transform incoming changes against their own; the server transforms late changes against every revision committed since the client's last sync, exactly the state machine in this lesson (Google Drive blog).
- Figma, 2019: multiplayer without OT. Figma judged OT too complex for a tree of design objects. Each document gets one server process over WebSockets that is the authority; properties merge last-writer-wins in the order the server receives them, child order uses fractional indexing, reparenting that would create a cycle is rejected, and clients that were offline download a fresh copy and reapply their edits (Figma).
Trade-offs
| Option | Strengths | Weaknesses | Choose when |
|---|---|---|---|
| Client-server OT | Small ops, simple server ordering, proven in Docs | Needs a live server per doc; weak offline story | Always-online collaborative editors (our choice) |
| CRDT (Yjs, Automerge) | Offline, peer-to-peer, no central orderer | Metadata and tombstones, harder rich-text semantics | Local-first apps, long offline periods |
| Paragraph locks | Trivial to implement | Users block each other | Structured forms, low concurrency |
| Last-writer-wins per document | No merge logic | Loses work | Never for shared text |
| Snapshot every N ops | Fast opens, bounded replay | Snapshot write cost, extra storage | Any long-lived document |
Leadership lens
- Build first: the transform function with a randomised convergence test (generate random concurrent ops, apply in both orders, assert equal results) and the client state machine. Every bug in this product eventually shows up as divergence, and that test catches most of them.
- Rollout: ship plain text with one owner per document and a single region first, then rich text, comments and presence, then multi-region placement.
- Ownership: an editor-core team owns the op model and client engine; a realtime team owns gateways, routing and session owners; a storage team owns the op log, snapshots and history. The op format is a shared contract and needs versioning from day one.
- Cost drivers: idle WebSocket connections (memory on gateways), op log storage before compaction, and cross-region replication.
- Design review question: how do we detect divergence in production? Answer: clients periodically send a hash of their document at a revision, and the server compares.
Check yourself
Why choose client-server OT over a CRDT for this editor?
Show answer
With one server ordering every operation, the transform only needs the simpler two-operation convergence property (TP1), and ops stay small. CRDTs need no central orderer and suit offline or peer-to-peer use, but carry per-character metadata and tombstones, so they fit local-first apps better than an always-online editor.
An old session owner pauses, its lease expires, and it wakes up after a new owner has taken over. Why can it not corrupt the document?
Show answer
Every edit is a conditional append of
(doc_id, rev)to the op log, so when both owners try to write the same next revision only one append succeeds. The loser discovers it is stale, drops its sessions and lets clients reconnect to the new owner.A session owner crashes. Why does no edit get lost or applied twice?
Show answer
An edit is acked only after it is durably appended to the op log, so the new owner rebuilds state from the latest snapshot plus the log tail. Clients resend their in-flight op with the same
(clientId, seq), and the server's dedupe returns the original ack instead of applying it again.Why are snapshots and compaction needed, based on the estimate of the raw op log?
Show answer
At about 300,000 ops per second averaged over the day and 100 bytes each, the raw log grows by about 2.6 TB a day, roughly 4.7 PB over five years. Snapshotting every 500 or so ops keeps opens to one blob read plus a short tail, and compaction keeps fine-grained ops only for 30 days.
Sources and further study
Primary
- Gentle and Kleppmann, "Collaborative Text Editing with Eg-walker" (2024): an event-graph algorithm that keeps OT's small steady state with CRDT-style merging.
- Kleppmann et al., "Interleaving anomalies in collaborative text editors" (PaPoC 2019): why convergence alone does not make a text CRDT usable.
- Neil Fraser, "Differential Synchronization": the diff-and-patch alternative to OT, from a Google engineer, used in MobWrite.
Systems docs
- Yjs documentation: shared types, providers and awareness (presence) in the most widely used JavaScript CRDT.
- Automerge documentation: a JSON-like CRDT with full history, built for local-first apps.
- Google Docs Editors Help: share files: the 100-open-tabs editing limit and what happens past it.
Books
- Designing Data-Intensive Applications, Martin Kleppmann, chapter 5 "Replication": multi-leader replication, with collaborative editing as the example, and conflict resolution including OT and CRDTs.
- Understanding Distributed Systems (2nd ed.), Roberto Vitillo, chapter "Coordination avoidance": broadcast and CRDTs as the way to converge without a central orderer.
Other curricula
- donnemartin/system-design-primer: lists "Design Google docs" and points to Neil Fraser's differential synchronization rather than OT, a useful counterpoint.
Watch
- CRDTs: The Hard Parts (Martin Kleppmann): opens with a clear explanation of operational transformation before showing where CRDTs differ and struggle.
- Design Google Docs | System Design Interview (Aced, formerly Exponent): a full mock interview of the same question.
- Google Docs System design, Part 1: Operational transformation (Tech Dummies): a whiteboard walkthrough of OT and differential synchronisation.
Free resources
- What's different about the new Google Docs: conflict resolution (Google Drive blog): Google's own description of how Docs applies operational transformation.
- How Figma's multiplayer technology works (Figma blog): a production design with a central server per document and a clear comparison with OT and CRDTs.
- Operational transformation (Wikipedia): background, the TP1 and TP2 properties, and the history of OT systems.
- Local-first software (Ink and Switch): the case for CRDT-based editors and their trade-offs against server-centric designs.
- CRDTs go brrr (Joseph Gentle): how modern CRDT implementations got fast enough for real text editing.