Engineering Atlas › System Design Interviews

Design a cloud file storage and sync service

20 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

Requirements

Functional

Non-functional

Out of scope: real-time co-editing of Docs, Sheets and Slides (covered in Design Google Docs), full-text search inside files, malware scanning internals, and the photo product.

Back-of-envelope

Assumed round numbers, not Google's figures.

API

POST /v1/blocks/check        { hashes: ["sha256:9f2c...", ...] }
                          -> { missing: ["sha256:9f2c..."] }
PUT  /v1/blocks/{hash}       body: up to 4 MiB of bytes; server re-hashes and rejects a mismatch
                          -> 201 Created

POST /v1/files/commit        { ns_id, path, base_rev, size, blocks: ["sha256:...", ...], client_mtime }
                          -> { status: "ok", rev: 88412 }
                           | { status: "need_blocks", missing: [...] }
                           | { status: "conflict", path: "plan (laptop conflicted copy).txt", rev: 88413 }
POST /v1/files/delete        { ns_id, path, base_rev } -> { status: "ok" | "stale", rev }
POST /v1/files/move          { ns_id, from, to, base_rev } -> { status, rev }

GET  /v1/changes?ns_id=..&cursor=88400&limit=500
                          -> { entries: [ { seq, path, deleted, rev, blocks } ], cursor: 88412, has_more: false }
GET  /v1/changes/wait?cursor=88412&timeout=90   (long-poll) -> { changed: true | false, backoff: 0 }

GET  /v1/blocks/{hash}       (CDN-fronted, signed URL, supports Range)
POST /v1/shares              { target_id, grantee, role: "viewer" | "commenter" | "editor" } -> { share_id }

commit is idempotent: replaying the same (path, base_rev, blocks) after a timeout returns the revision already created. base_rev is 0 for a new file. The changes feed follows the same shape as Google Drive's public Changes API, where a startPageToken plays the role of the cursor.

Data model

namespaces (ns_id) -> owner, kind (user_root | shared_folder), head_seq. A namespace is a subtree with its own journal. Each user has a root namespace, and each shared folder is a namespace mounted into every member's tree, so sharing costs one mount row per member.

journal primary key (ns_id, seq), partitioned by ns_id. Columns: path, file_id, rev, deleted, blocks_ref. seq is gap-free and increases by one per commit within a namespace. A device's cursor is simply the last seq it applied. Partitioning by namespace keeps each commit a single-shard transaction.

files (ns_id, path) -> file_id, rev, size, blocks_ref, mtime, in the same shard as the namespace's journal so the file row and the journal row are written in one transaction. A secondary index on (ns_id, parent_path) serves folder listings.

versions (file_id, rev) -> blocks_ref, size, author_device, created_at. History for restore; a retention job deletes versions past the plan's window.

block_lists (blocks_ref) -> [hash, hash, ...]. Large files have thousands of hashes, so they are stored once and referenced.

blocks (hash) -> size, storage_location, refcount, partitioned by hash. The hash is uniformly distributed, so this table shards evenly with no hot spots. refcount (or a periodic mark-and-sweep) decides when a block can be deleted.

mounts (user_id, ns_id) -> mount_path, role. Read on login and on every permission check, so it is cached heavily.

High-level design

flowchart LR
  C["Desktop / mobile client"] -->|"check, commit, changes"| API["API gateway + auth"]
  API --> META["Metadata service"]
  META --> DB[("Sharded metadata DB by ns_id")]
  C -->|"PUT / GET blocks"| BS["Block service"]
  BS --> BLOB[("Blob store, erasure coded")]
  BLOB --> CDN["CDN for downloads"]
  META -->|"journal append"| NOTIF["Notification service"]
  NOTIF -->|"wake long-poll / push"| C
  META --> Q["Event bus"]
  Q --> W["Workers: thumbnails, index, virus scan, GC"]

Write path.

  1. The desktop client notices a change through the OS file watcher, waits for the file to stop changing, then splits it into blocks and hashes each with SHA-256.
  2. It calls blocks/check with the hash list. The block service looks the hashes up and returns the ones it does not have.
  3. The client uploads only the missing blocks, several in parallel. The block service re-hashes each one, writes it durably to the blob store, and only then records it in blocks.
  4. The client calls commit with the full block list and its base_rev. In one transaction on the namespace's shard, the metadata service checks that every block exists, checks base_rev against the file's current rev, bumps head_seq, and writes the file row, the version row and the journal row.
  5. After commit, the metadata service tells the notification service that the namespace advanced, and publishes an event for thumbnails, indexing and scanning.

Read path.

  1. Each online device holds a long-poll on changes/wait with its cursor for each namespace it mounts (or one connection that multiplexes them).
  2. When the namespace's head_seq passes the cursor, the poll returns changed: true. The client calls changes to read journal entries after its cursor in pages.
  3. For each entry, the client downloads only the blocks missing from its local cache, writes the file to a temp path, fsyncs and renames it into place.
  4. The client advances its cursor only after the files are on disk. A crash replays from the old cursor, and since applying an entry is idempotent, nothing breaks.

Deep dives

1. Chunking and deduplication

The block is the unit of upload, download, resume and dedup, so its size and boundaries matter. Fixed-size blocks are simple: offset divided by 4 MiB gives the block index, and Dropbox has long used 4 MB blocks; its public content_hash is still a SHA-256 over the SHA-256 of each 4 MiB block (Dropbox API). The weakness shows when bytes are inserted near the start of a file: every later boundary shifts, and every block hash changes, so a one-byte insert re-uploads the whole file.

Content-defined chunking fixes that. A rolling hash slides over the bytes, and a boundary is cut wherever the hash's low bits are all zero, subject to a minimum and maximum size. LBFS used Rabin fingerprints this way with about 8 KB average chunks (LBFS, SOSP 2001), and FastCDC uses a cheaper gear hash (FastCDC, ATC 2016). Boundaries now follow content, so an insert changes one or two chunks and the rest re-align. The cost is CPU on the client and many more chunks per file, which means more metadata rows and more hash lookups.

I would pick content-defined chunks with a large average size (around 1 to 4 MB), so a 1 GB file has a few hundred entries, not a hundred thousand.

Dedup across users saves money but leaks information: if uploading a block is instant, an attacker learns that someone already stores that exact content. Dedup only within an account or organisation, or always demand the upload and dedup silently afterwards. This is the duplicates problem shape; git's object store and container image layers solve it with the same content addressing.

2. The journal, cursors and notifications

Options: push full change events through a broker, have devices list folders and diff, or keep a durable, ordered journal per namespace read by cursor. Pushed events are lost while a device is offline; diffing costs O(size of tree) per poll. The journal costs O(changes) and works the same after five seconds or five weeks.

Per namespace, not per user, because a shared folder's commit is then written once and read by every member through their mount. A per-user journal needs fan-out on write, which hurts for a folder shared with a 10,000-person company.

Notifications carry no data, only "namespace X is now at seq N", so losing one is harmless. Drive's own changes.watch channels work the same way: the push says only that changes exist, and the client then calls changes.list (Drive API). The notification tier is a stateless fleet of long-poll holders. When the cursor is older than the journal's retention (say 90 days after compaction), the server answers reset and the client does a full listing, reconciling by hash so it does not re-download what it already has.

3. Conflicts and the commit contract

Two devices edit plan.txt while one is offline. Options: last writer wins (simple, silently destroys work), merge (impossible in general for binary formats), or optimistic concurrency with conflicted copies. The commit's base_rev is the compare-and-set token. If it does not match the current rev, the server does not reject the bytes; it commits them under a new name like plan (laptop conflicted copy).txt and returns that path. Both versions exist, and the human decides.

A delete based on a stale revision returns stale instead of deleting the newer edit. A move across namespaces (into a shared folder) is a copy then a delete, made safe by idempotent operation ids. Folder renames are the expensive case: renaming a folder with 100,000 files rewrites 100,000 path rows unless paths are stored as parent-pointer trees with file_id keys. I would store the tree by id and derive paths, at the cost of a recursive lookup for path-based API calls, which a cache absorbs.

Down the stack

Layer 1The contract

  • Atomic commit. A commit either creates exactly one new revision whose every block is durably stored, or changes nothing. No reader can observe a revision with a missing block.
  • Ordered journal. Within a namespace, seq values are gap-free and assigned in commit order. Reading from cursor N returns every committed change with seq > N, exactly once, in order.
  • Compare-and-set. A commit with base_rev equal to the current revision succeeds as an update; any other base_rev produces a conflicted copy, never an overwrite. Replaying an identical commit returns the original result.
  • Eventual visibility. A change committed on one device is visible to every online device with read access within seconds, and to offline devices on their next sync, as long as the cursor is within retention.

Layer 2The mechanism

Dropbox's sync engine, rewritten in Rust and called Nucleus (Dropbox Tech), models sync as three trees: the remote tree (what the server says), the local tree (what is on disk) and the synced tree (the last state both agreed on) (Dropbox Tech). Comparing local to synced finds local edits to upload; comparing remote to synced finds server edits to apply; a path that changed in both is a conflict. That base tree is the same idea as the merge base in a three-way merge.

On the server, Dropbox keeps metadata in Edgestore over sharded MySQL. Growing Edgestore meant splitting every shard in two, so in 2022 Dropbox put Panda underneath it: a range-sharded, transactional key-value layer that moves ranges of about 100 GB between nodes to add capacity gradually (Dropbox Tech). Blocks live in Magic Pocket, its own blob store, which groups blocks of up to 4 MB into 1 GB buckets, stores each block in at least two zones, and erasure-codes older data inside cells of about 50 PB (Dropbox Tech). Clients detect local changes through inotify on Linux, FSEvents on macOS and ReadDirectoryChangesW on Windows, and keep their own state in a local SQLite database.

Google Drive's public API has the same shape: resumable uploads (uploadType=resumable) return a session URI that accepts chunks in multiples of 256 KB, answers 308 Resume Incomplete until the last one, and expires after a week (Drive API), and the Changes API hands out a startPageToken cursor, with optional webhook channels as the wake-up signal.

Layer 3The machine

  • Hashing. SHA-256 with the x86 SHA extensions runs at roughly 1.5 to 2 GB/s per core, so hashing a 4 MiB block takes 2 to 3 ms. Re-hashing a 10 GB folder on startup is several seconds of CPU, which is why clients cache (inode, size, mtime) -> hashes and re-hash only files whose stat changed.
  • File watching. inotify needs one watch per directory, and the per-user limit (fs.inotify.max_user_watches) defaulted to 8,192 from 2005 until Linux 5.11, which scales it with RAM between 8,192 and 1,048,576 (kernelnewbies, patch). Older kernels and small machines still hit it, so a large tree can silently outgrow it. Clients must handle overflow events (IN_Q_OVERFLOW) by falling back to a full scan.
  • Safe local writes. Applying a remote change writes to a temp file, fsyncs it, renames it over the target and fsyncs the directory. A plain overwrite risks a torn file after a crash, and the user blames the sync product.
  • Blob store disks. A hard drive does about 100 to 200 random IOPS but well over 200 MB/s sequentially. Writing each 4 MiB block as its own file would waste seeks and inode space, so blocks are appended into extents of around a gigabyte and located by (extent, offset).

Why it's built this way

  1. Files are large and change in small places, so bytes are split into blocks and only changed blocks move.
  2. Blocks are identified by their hash, so the same content is stored once and an upload interrupted at any point resumes by asking what is missing.
  3. Blocks can be uploaded in any order, so the moment of truth must be a separate, small metadata commit that only succeeds once every block is durable.
  4. Devices are offline for unpredictable lengths of time, so the commit also appends to an ordered journal that any device can replay from its cursor.
  5. Offline devices make concurrent edits inevitable, and binary files cannot be merged, so the commit is a compare-and-set that keeps the losing version as a copy.

Failure modes

In the wild

Trade-offs

OptionStrengthsWeaknessesChoose when
Fixed-size blocksSimple, fast, easy offset mathInserts shift every later blockMostly append-only or whole-file rewrites
Content-defined chunksInserts change only nearby chunksMore CPU and more metadataLarge files edited in the middle
Cross-user dedupLarge storage savingsLeaks existence of contentPublic or enterprise-internal data
Per-namespace journalOne write per change, shared folders are cheapClients track one cursor per mountShared folders are common
Per-user journal with fan-outOne cursor per deviceWrite amplification for big sharesMostly private files
Conflicted copiesNever loses dataUsers must resolve by handBinary and arbitrary files
Last writer winsNo user-facing conflictsSilently drops editsNever for user files

Leadership lens

Check yourself

  1. Two devices edit plan.txt while one is offline. What happens when the second one commits, and why not last writer wins?

    Show answer

    Its base_rev no longer matches the current rev, so the server keeps the bytes but commits them as a conflicted copy such as plan (laptop conflicted copy).txt. Last writer wins would silently destroy work, and binary files cannot be merged in general, so both versions are kept and the human decides.

  2. Why prefer content-defined chunking over fixed-size blocks, and what does it cost?

    Show answer

    With fixed blocks, inserting bytes near the start shifts every later boundary and changes every block hash, so the whole file is re-uploaded. A rolling hash cuts boundaries by content, so an insert changes only one or two chunks, at the cost of client CPU and more chunks, which means more metadata rows and hash lookups.

  3. Why is the journal kept per namespace rather than per user?

    Show answer

    A commit to a shared folder is then written once and read by every member through their mount. A per-user journal needs fan-out on write, which hurts badly for a folder shared with a 10,000-person company, though it would give each device a single cursor.

  4. How much new block storage does the design need over five years, and why does dedup matter to that number?

    Show answer

    800M commits a day at 1 MB each, with only 40% new bytes after dedup and delta upload, is about 320 TB a day, or roughly 584 PB logical over five years and about 820 PB raw with erasure coding. Without block-level dedup that daily figure would be 2.5 times larger, so dedup and retention policy move the storage bill more than any compute work.

Sources and further study

Primary

Systems docs

Books

Other curricula

Watch

Free resources

Open in the interactive app: practice, quiz, mock interview →