Amazon S3 for system design interviews

In one minute

• Amazon S3 is an object store: you PUT and GET whole objects by bucket and key over HTTPS. There are no in-place edits or real directories.
• It is designed for eleven nines of durability by storing data redundantly across at least three availability zones, and since December 2020 reads, writes and lists are strongly consistent (AWS).
• Throughput comes from parallelism: each prefix supports at least 3,500 writes and 5,500 reads per second and splits under load; large objects move as parallel parts and ranged GETs.
• First-byte latency is tens to low hundreds of milliseconds, so S3 stores and streams data; it is not a low-latency database.
• In interviews, S3 is the default for media, backups, logs and data lakes. Keep metadata in a database and the bytes in S3.

How it works

Buckets, keys and objects

A bucket is a namespace in one region. A key is any UTF-8 string up to 1,024 bytes; photos/2026/a.jpg is one flat key, and "folders" are a listing convenience using a delimiter. An object is immutable once written: a PUT to an existing key replaces it atomically, and with versioning enabled the old version is kept. Objects can be up to 50 TB (raised from 5 TB in December 2025; AWS), but a single PUT is limited to 5 GB.

Multipart upload

Large objects are uploaded in parts. CreateMultipartUpload returns an UploadId. The client uploads parts numbered 1 to 10,000, each 5 MiB to 5 GiB except the last, in any order and in parallel, and each UploadPart returns that part's ETag (AWS limits). Because the part count is capped, part size caps object size: 16 MiB parts top out near 156 GiB, and only parts close to 5 GiB reach the 48.8 TiB maximum, so pick the part size from the largest file you expect. CompleteMultipartUpload sends the list of part numbers and ETags in ascending order, and S3 stitches them into one object atomically. Until completion or AbortMultipartUpload, the parts are stored and billed but invisible, which is why every bucket should have a lifecycle rule that aborts incomplete uploads.

Reading, listing and conditional writes

GET accepts a Range header, so clients download large objects as parallel byte ranges. ListObjectsV2 returns up to 1,000 keys per page in lexicographic order. Conditional writes (If-None-Match: * to create only if absent, If-Match: <etag> to replace only an expected version) let S3 act as a simple coordination point, for example for table-format commit files. When several conditional writers race, the first to finish wins and the rest get 412 Precondition Failed; a concurrent delete can produce 409 Conflict, and for multipart uploads that means starting the upload again (AWS docs).

Storage classes and lifecycle

The same API fronts cost tiers from Standard through Infrequent Access to Glacier archive classes. Lifecycle rules move or expire objects by age or prefix; colder tiers trade cheaper storage for retrieval fees and, for archives, hours of restore time.

Down the stack

Diagram

sequenceDiagram
    participant C as Client
    participant F as Front end
    participant I as Index (metadata)
    participant S as Storage nodes (EC shards)
    C->>F: CreateMultipartUpload
    F-->>C: UploadId
    par parts in parallel
        C->>F: UploadPart 1 (16 MB)
        F->>S: write k+m shards across AZs
        F-->>C: ETag 1
    and
        C->>F: UploadPart 2 (16 MB)
        F->>S: write k+m shards across AZs
        F-->>C: ETag 2
    end
    C->>F: Complete [(1,ETag1),(2,ETag2)]
    F->>I: commit object manifest atomically
    F-->>C: ETag "md5(md5s)-2"

Worked example

A video service ingests 50,000 uploads a day, averaging 1.2 GB, and serves 30 million plays a day of transcoded renditions through a CDN.

Ingest is 60 TB a day. Clients upload directly to S3 with presigned multipart URLs, so bytes never touch application servers. With 16 MB parts a 1.2 GB file is 75 parts; the upload bandwidth is the same, but a dropped part is retried alone instead of restarting the file. A lifecycle rule aborts incomplete uploads after 3 days.

Keys are raw/<upload_id>/source.mp4 and renditions/<video_id>/<height>/<segment>.ts; random IDs spread writes across the key space. With 2-second segments, a 10-minute video in 5 renditions is 1,500 objects, so 75 million PUTs a day, about $375 a day in requests alone. Packing segments into one fMP4 file per rendition with byte-range reads cuts PUTs 300-fold.

After a year storage is about 22 PB of originals plus renditions. Originals move to Glacier Flexible Retrieval after 30 days, cutting their cost by roughly 80 percent. The CDN absorbs most reads, so S3 sees only a few thousand GETs per second of cache misses.

In the wild

• AWS, 2017: one mistyped command takes S3 down in us-east-1. While debugging billing, an engineer meant to remove a few servers, but a mistyped argument removed many more, including capacity behind the index (metadata) and placement subsystems. Both needed a full restart that had not been done in years at that scale, and S3 in the region was unavailable for about four hours, taking EC2 launches, EBS restores, Lambda and AWS's own status page with it. The tool now removes capacity slowly and refuses to go below minimums, and work on smaller cells was accelerated (AWS).
• Amazon S3, 2008: one flipped bit in gossip. On 20 July a few internal server-state messages carried a single corrupted bit that still parsed. S3 checksummed customer objects but not this gossip traffic, so the bad state spread until servers spent nearly all their time gossiping. Errors began at 8:40am PDT; AWS stopped server-to-server communication, cleared state and recovered by 4:58pm, then added checksums to the gossip messages (AWS, archived).
• GitLab, 2021: abandoned multipart uploads keep billing. A self-managed customer counted more than 67,276 incomplete multipart uploads created in three days. GitLab had cleaned up its temporary files but never completed or aborted the uploads, so their parts kept accruing storage charges; the advised fix was an AbortIncompleteMultipartUpload lifecycle rule (GitLab).

Trade-offs

Pitfalls in production

• Incomplete multipart uploads. Abandoned parts are billed but invisible. Add an AbortIncompleteMultipartUpload lifecycle rule everywhere.
• Millions of tiny objects. Request overhead dominates cost. Batch small records into larger files.
• Listing as a query engine. Scanning 1 billion keys is a million LIST calls. Keep an index in a database or use S3 Inventory.
• Sudden traffic to one prefix. Expect 503 SlowDown until S3 repartitions. Spread keys and retry with backoff.
• Public buckets and leaked presigned URLs. Since April 2023 new buckets start with Block Public Access on and ACLs disabled (AWS), and since January 2023 every new object is encrypted with SSE-S3 (AWS docs), but older buckets keep whatever they had. Audit them, use short presigned expiries, and serve public content via a CDN.

Leadership lens

• Decide early what lives in S3 (bytes) versus a database (metadata, permissions), and keep LIST off user-facing paths.
• Request counts, lifecycle tiers and cross-region or internet egress usually dominate the bill more than raw storage; model all four.
• Use versioning plus Object Lock or replication for anything that would be a disaster to delete, and test restores.
• Presigned direct uploads keep large bytes off application servers; make them the default pattern.

Check yourself

1. Why is the ETag of a multipart object not the MD5 of the whole file?

   Show answer

   S3 computes it from the parts as the MD5 of the concatenated part digests plus a part count, so it never has to rehash the assembled object.

2. A team writes all objects under logs/2026-10-15/ and sees 503 SlowDown at 6,000 writes per second. What is happening?

   Show answer

   All writes hit one key range above its per-prefix rate; S3 will split the range over time, but spreading keys (such as hashed prefixes) avoids the hot spot.

3. Why does erasure coding allow both lower cost and higher durability than three replicas?

   Show answer

   k data plus m parity shards survive any m losses at (k+m)/k overhead, 1.8x for 5+4, versus 3x for replicas that survive two.

4. Can two clients safely use S3 to elect a leader?

   Show answer

   Yes: each PUTs the same key with If-None-Match: * and only one succeeds, though lease expiry still needs care.

Sources and further study

Primary

• Vogels, Diving deep on S3 consistency (2021): how the witness and cache-coherency protocol made S3 strongly consistent without losing its cache.
• AWS: Summary of the Amazon S3 service disruption in us-east-1 (2017): the clearest public description of S3's index and placement subsystems.

Systems docs

• Amazon S3 multipart upload limits: part counts, part sizes and the maximum object size.
• Amazon S3 conditional writes: If-None-Match, If-Match, and the 412 and 409 cases.
• Data protection in Amazon S3: the durability and availability design targets per storage class.
• Amazon S3 default encryption FAQ: SSE-S3 on every new object since January 2023.

Books

• System Design Interview, Volume 2, Alex Xu and Sahn Lam, Ch 9 "S3-like Object Storage": metadata versus data stores, erasure coding, multipart upload and garbage collection in interview form.
• Designing Data-Intensive Applications (1st ed.), Martin Kleppmann, Ch 10 "Batch Processing": distributed filesystems and object stores as the shared-nothing storage under batch jobs.

Other curricula

• Azure Architecture Center: Valet Key pattern: the general form of presigned URLs, with short scopes and expiries and the warning that a key usually cannot cap upload size.
• donnemartin/system-design-primer: has no object-storage chapter; its SQL tuning advice is to keep large blobs out of the database and store their location instead.

Watch

• FAST '23: Building and Operating a Pretty Big Storage System (USENIX): Andy Warfield on S3's drives, heat management, durability and ShardStore.
• How AWS S3 is built (The Pragmatic Engineer): an interview on S3's architecture, consistency change and engineering culture.

Free resources

• Building and operating a pretty big storage system called S3: Andy Warfield's essay on drives, heat and durability at S3 scale.
• Uploading and copying objects using multipart upload: official limits, flow and part rules.
• Best practices design patterns: optimizing Amazon S3 performance: per-prefix rates, parallelism and ranged GETs.
• Amazon S3 strong consistency: what the consistency model guarantees.
• Checking object integrity: checksums, ETags and multipart checksum calculation.
• Using lightweight formal methods to validate a key-value storage node in Amazon S3: the ShardStore paper.