Engineering Atlas › System Design Interviews

Design a social news feed

20 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

Requirements

Functional:

Non-functional:

Out of scope: the ranking model itself (we treat it as a service that scores candidates), ads insertion, media transcoding, messaging, search and privacy-setting edge cases beyond a simple audience check.

Back-of-envelope

Assume 2 billion monthly users and 1 billion daily active users. These are round planning numbers, not Meta's published figures.

Reads: each DAU opens the feed about 10 times a day and each open fetches about 2 pages. That is 1B x 20 = 20B page requests per day. Divide by 86,400 seconds: about 230,000 requests per second on average. Peak at 3x average gives roughly 700,000 feed reads per second.

Writes: assume 1 in 5 DAUs posts once a day, so 200M posts per day, or about 2,300 per second on average and 7,000 per second at peak.

Fan-out: assume the median author has about 200 followers, but the mean is higher because of the long tail; say 300. If we fanned out every post we would write 200M x 300 = 60B inbox entries per day, about 700,000 per second. That is comparable to the read rate, which is why fan-out on write is affordable for most authors. A single post from an account with 100M followers alone would be 100M writes, which is why it is not affordable for everyone.

Storage for posts: about 1 KB of text and metadata per post, so 200M x 1 KB = 200 GB per day, about 73 TB per year, and about 365 TB over five years before replication. With 3 replicas, call it 1.1 PB. Media is far larger but sits in blob storage and a CDN, sized separately.

Storage for inboxes: each entry is a post ID (8 bytes), author ID (8 bytes), timestamp (8 bytes) and a few flags, so roughly 32 bytes. Capping inboxes at 500 entries for active users gives 1B x 500 x 32 B = 16 TB. Spread across a memory tier with replication, that is a few hundred cache servers with a few hundred GB each, which is expensive but routine at this scale.

Bandwidth: a feed page of 20 posts is about 20 KB of JSON before media. At 700,000 pages per second, that is 14 GB/s, or about 110 Gbit/s of API egress at peak. Images and video come from the CDN.

API

POST /v1/posts
  body: { text, media_ids[], audience: "friends" | "public" }
  headers: Idempotency-Key
  -> 201 { post_id, created_at }

GET /v1/feed?cursor=<opaque>&limit=20
  -> 200 { items: [ { post_id, author{id,name,avatar}, text, media[],
                      like_count, comment_count, viewer_liked } ],
           next_cursor }

POST /v1/follows        body: { followee_id }   -> 204
DELETE /v1/follows/{followee_id}                -> 204

The cursor encodes the ranked snapshot ID and an offset, so scrolling does not reshuffle items already seen. The idempotency key stops a retried POST from creating a duplicate post.

Data model

The pattern is that each table is partitioned by the ID the hottest query starts from.

High-level design

flowchart LR
  C["Client"] --> GW["API gateway"]
  GW --> PS["Post service"]
  PS --> PDB["Posts + outbox store"]
  PS --> Q["Post-created log"]
  Q --> FO["Fan-out workers"]
  FO --> G["Follow graph"]
  FO --> IN["Feed inbox cache"]
  GW --> FS["Feed service"]
  FS --> IN
  FS --> G
  FS --> PDB
  FS --> RK["Ranking service"]
  FS --> HY["Hydration: posts, users, counters"]
  HY --> CDN["CDN for media"]

Write path: the client sends the post with an idempotency key. The post service writes the post row and the outbox entry, then appends a PostCreated event to a durable log partitioned by author. It acknowledges the client at this point, so post latency does not depend on fan-out. Fan-out workers consume the log, look up the author's follower count, and branch. If the author is below the celebrity threshold, they page through the follower list in chunks of about 1,000, filter out inactive followers, and push the post ID into each follower's inbox with a pipelined batch write. If the author is above the threshold, they do nothing beyond the outbox write already made.

Read path: the feed service loads the viewer's inbox (one key, one round trip). It loads the viewer's following list filtered to celebrities, a set that is usually small and is cached per user, and reads the latest few dozen posts from each celebrity outbox in parallel. It merges the two candidate sets, drops duplicates and posts the viewer has already seen, and sends a few hundred candidates to ranking. The top page is hydrated (post bodies, author profiles, counters, viewer-liked flags) via batched multi-gets and returned with a cursor. The ranked list is cached briefly so page 2 is a cache read.

Deep dives

Fan-out on write vs read, and the threshold

Pure push makes reads one key lookup, but a 100M-follower post becomes 100M writes and delays delivery for everyone queued behind it. Pure pull makes writes trivial, but a viewer following 500 accounts triggers 500 outbox reads per feed load, times 700,000 loads per second, which is hundreds of millions of reads per second.

The hybrid takes push for the long tail and pull for the head. The threshold is chosen from the read side: pulled authors add per-read work, so you want the number of pulled authors per viewer to be small (say under 20 to 50). A threshold in the range of tens of thousands to a few hundred thousand followers usually achieves that while removing the worst fan-out jobs. Treat the exact number as a tunable, set from production histograms. Production systems have sat at both ends of this spectrum. Twitter, in 2012, fanned every tweet out into Redis home timelines capped at 800 entries, served about 300K timeline reads per second against about 5K tweets per second, and accepted that a tweet from an account with 31 million followers could take up to 5 minutes to reach everyone (High Scalability, from Raffi Krikorian's QCon talk). Facebook's News Feed went the other way. Its Multifeed system builds the feed at read time: an aggregator takes the request, queries leaf servers that index friends' recent actions in memory, then ranks the results. A tailer streams new actions into the leaves, so there is no per-viewer inbox at all (Engineering at Meta). That works because Facebook ranks heavily anyway, so it must touch many candidates per read. Say this in an interview: the hybrid is a sound default, but pull over an in-memory index is a legitimate answer when ranking dominates.

Edge cases to call out: an author crossing the threshold should not double-deliver (dedupe by post ID at merge), and pages run by teams with huge audiences but low posting rates could stay on push, so the decision can also weigh posting frequency. Yahoo's Feeding Frenzy paper made this precise: total cost is minimized by deciding push or pull separately for each producer and consumer pair, by comparing how often the producer posts with how often the consumer views the feed (Silberstein et al., SIGMOD 2010). A follower threshold is a coarse version of that rule.

Inactive users and inbox lifecycle

Most fan-out writes go to people who will not look today. Track last_active_at, and skip inbox writes for followers inactive for, say, 14 days. When such a user returns, the feed service sees a missing or stale inbox and rebuilds it by pulling from the outboxes of everyone they follow, bounded to the most recent posts, then writes the result as the new inbox. The first load is slower (it may take a second), so show a cached or partial result immediately and fill in. This one policy can cut fan-out volume by a large fraction because daily actives are a minority of total accounts.

Ranking, pagination and freshness

Ranking needs features (author affinity, post age, engagement so far, content type) and runs a model per candidate. Do it in stages. Meta describes three passes for Facebook: a lightweight model narrows the eligible pool (more than 1,000 posts per user per day on average) to about 500, the main pass scores each of those independently as a weighted sum of predicted actions such as like, comment and share, and a final contextual pass applies rules such as content-type diversity (Engineering at Meta). Pagination over a ranked feed cannot use "posts older than timestamp T" because ranked order is not time order. Instead, snapshot the ranked list under a session ID with a short TTL (a few minutes) and page by offset into it. New posts that arrive during scrolling appear on the next pull-to-refresh, which matches user expectations and keeps pages stable.

Variant: a news aggregation service

A news aggregator looks like a feed but inverts the producer side. In a social feed, authors are your own users and every post arrives through your write API. In an aggregator the content lives on roughly 100k publisher sites you do not control, so you must go and get it, and the unit shown to readers is a story cluster, not a post.

Ingestion replaces the write path. Freshness is now a crawling problem. A fixed poll interval is wrong at both ends, so estimate each feed's publish rate with a moving average and set the next poll so about one new item is expected, clamped between 30 seconds and 2 hours, with jitter. Conditional GETs (ETag, If-Modified-Since, RFC 9110 section 13) make an empty poll cost a few hundred bytes. Where publishers support WebSub, the hub pushes instead, but subscriptions carry a lease that must be renewed, so a catch-up poll is still needed. News sitemaps (last two days, at most 1,000 entries, Google Search Central) cover sites with no usable feed. A per-domain token bucket keeps the crawler polite.

Deduplication in two tiers. An exact content hash drops syndicated copies. For lightly edited copies, shingle the text and compute MinHash signatures, then use locality-sensitive hashing so each arrival does a fixed number of bucket lookups instead of comparing against the last three days of articles. With 32 bands of 4 rows, a pair of true similarity 0.5 becomes a candidate about 87% of the time and a pair at 0.15 under 2% (Lee et al. show the same pipeline at corpus scale). Candidates are verified before merging.

Clustering is the data model change. Different articles about one event are grouped by shared entities plus embedding similarity against stories updated in the last 48 hours. When a bridging article joins two clusters, the older ID survives and the other becomes an alias, which keeps old links working.

Ranking is per story and mostly precomputed. Score a cluster by authority-weighted source count on a logarithmic curve (the 40th outlet adds little), a freshness term that decays from the newest article, and a velocity term for sources added recently. Compute a few hundred candidates per region and topic each minute, then rerank per user in memory using a topic-affinity vector. This replaces the per-user inbox: with 50M users but a shared candidate pool, there is nothing to fan out.

Numbers shift. Writes are tiny (about 17 new articles a second after dedup, around 90 in a burst) while reads peak near 17k feed loads a second, so the read cache and CDN-served thumbnails dominate cost, not storage writes.

Down the stack

Layer 1The contract

A feed read returns, within about 300 ms, a page of posts the viewer is allowed to see, without duplicates within a session, from a snapshot that is at most a few seconds behind for active users. Posts are durable once acknowledged. Delivery is at-least-once into inboxes, so the merge must be idempotent by post ID. A deleted post must never be shown after hydration, even if its ID is still in inboxes, which means hydration is the authority and inbox entries are hints.

Layer 2The mechanism

The PostCreated log is partitioned by author_id, so a single author's posts are fanned out in order. Each fan-out worker holds a cursor into the follower list (author_id, last follower_id) and checkpoints it with the log offset, so a crashed worker resumes mid-list and re-sends at most one chunk. Inbox insert is a sorted-set add keyed by post ID (score is time or a first-pass score), followed by a trim to the cap, sent as one pipelined batch per destination shard. In Redis terms the inbox is a sorted set, which is a skip list plus a hash table, so each ZADD is O(log N) and reads come back already ordered (Redis docs). The read-time merge is a k-way merge of already-sorted lists (inbox plus N celebrity outboxes) using a heap, which costs O(total log k) and stops after enough candidates.

Layer 3The machine

An inbox read is one network round trip to a memory store: about 100 to 300 microseconds inside a data centre, plus serialising about 16 KB (500 entries x 32 bytes), which fits in roughly a dozen 1,460-byte TCP segments. Fan-out writes are batched because the per-command cost of a Redis-style server is dominated by syscalls and parsing, not by the data structure; pipelining 1,000 inserts into one write and one epoll_wait cycle can raise throughput by an order of magnitude. Hydration is the expensive part of a read: 20 posts times several stores. Batched multi-gets keep it to one round trip per store, issued in parallel, so the tail latency is set by the slowest shard. Ranking models run on CPUs or accelerators with feature vectors fetched from a cache; that step often dominates the 300 ms budget.

Why it's built this way

  • Reads are about 100 times more frequent than writes, so precompute at write time.
  • Follower counts follow a power law, so precomputing for the head is unbounded work; pull those instead.
  • Memory is expensive, so inboxes hold IDs only, are capped, and are skipped for inactive users.
  • IDs in inboxes can go stale (deletes, privacy changes), so hydration re-checks every item and the inbox stays a cache.
  • Ranked order is not time order, so pagination uses a snapshot rather than a timestamp cursor.

Failure modes

In the wild

Trade-offs

OptionStrengthsWeaknessesChoose when
Fan-out on writeOne-key reads, fast feedHuge writes for big accounts, wasted on inactive usersMost authors have modest audiences
Fan-out on readCheap writes, always freshMany reads per load, high read latencySmall scale, or few follows per user
Hybrid with thresholdBounded work on both sidesTwo code paths, dedupe and threshold tuningPower-law follower graphs at scale
Inbox in memory storeSub-millisecond readsCost per GB, needs rebuild pathHot, latency-critical reads
Inbox in LSM storeCheap, durableHigher tail latencyLong-retention or archive feeds

Leadership lens

Check yourself

  1. Why does the design use a hybrid of fan-out on write and fan-out on read instead of picking one?

    Show answer

    Pure push turns a 100M-follower post into 100M writes, while pure pull makes a viewer following 500 accounts do 500 outbox reads per load at 700,000 loads per second. Pushing for the long tail and pulling for authors above a follower threshold bounds work on both sides, at the cost of two code paths, dedupe and threshold tuning.

  2. An inbox store shard is lost. What happens to the affected users' feeds?

    Show answer

    Inboxes are a cache, so those users fall back to a pull-based rebuild from outboxes and the follow graph until the shard is restored. It is slow but correct, because posts, outboxes and the graph are the durable state.

  3. Why can't a ranked feed paginate with "posts older than timestamp T", and what is used instead?

    Show answer

    Ranked order is not time order, so a timestamp cursor would skip or repeat items. The ranked list is snapshotted under a session ID with a short TTL and paged by offset, so scrolling stays stable and new posts appear on the next refresh.

  4. How large is the feed inbox tier, and which policies keep it affordable?

    Show answer

    Each entry is about 32 bytes, so 1B users with inboxes capped at 500 entries is about 16 TB in memory. Storing only IDs, capping inbox length and skipping inboxes for users inactive for weeks are the biggest levers on that memory cost.

Sources and further study

Primary

Systems docs

Books

Other curricula

Watch

Free resources

Open in the interactive app: practice, quiz, mock interview →