Engineering Atlas › System Design Fundamentals

Keeping cached data fresh

12 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

How it works

TTL only

Each entry expires after a fixed time. No writer needs to know the cache exists, and staleness is bounded by the TTL. It suits data where minutes of staleness are acceptable (recommendations, counts, profile pictures). The cost is a miss on every expiry even when nothing changed, and a hard ceiling on freshness.

Invalidate on write

In the cache-aside pattern, the application writes the database and then deletes the cache key, so the next reader repopulates it. Deleting is preferred over updating the cache, because two concurrent writers updating the cache can apply in the opposite order from their database commits, leaving the cache permanently wrong. Deleting converges: whoever reads next gets the committed value.

The stale-set race

Deleting is still not enough. Reader R misses, reads value v1 from the database, then pauses (GC, slow network). Writer W commits v2 and deletes the key. R resumes and sets v1 into the cache. The cache now holds v1 with a full TTL, and nothing will fix it until expiry. Fixes:

  1. Leases. On a miss the cache hands the reader a token. A delete invalidates outstanding tokens, so R's late set is rejected.
  2. Versioned values. Store a version (row version or commit timestamp) with the value, and only accept sets with a newer version than what the cache has seen, including deletes as tombstones.
  3. Delayed double delete. Delete again a few hundred milliseconds after the write. Cheap, but probabilistic.
  4. Delete with a hold-off. Memcached deletes can carry a hold-off time during which add for that key fails, so a late fill bounces. Facebook issued every delete to a warming cluster with a two-second hold-off (Facebook NSDI 2013).

Invalidation from the database log

Application-level deletes are lost if the process crashes between commit and delete, and every write path must remember to do them. Tailing the database's change log (MySQL binlog, PostgreSQL logical replication) and issuing deletes from it guarantees every committed change produces an invalidation, in commit order, at the cost of a pipeline with its own lag. Debezium's Postgres connector, for example, reads logical decoding output through the built-in pgoutput plugin and a replication slot it creates for itself (Debezium docs).

Measure staleness, do not assume it

Invalidation bugs hide in rare error paths, so mature teams measure consistency from outside. Meta's Polaris acts like a cache server to receive invalidations, then queries replicas and reports keys still stale after 1, 5 or 10 minutes. With it, Meta raised TAO's cache consistency from six nines to ten nines of writes consistent within five minutes. One bug it caught: an invalidation hit a transient error, the error handler dropped the key only if the cached version was older, the stale value carried the newest version, and so it stayed forever (Meta engineering).

Many layers and regions

Real systems cache in several places: in-process, a shared cache cluster, a CDN, the browser. Each layer needs its own mechanism, and the outermost (browser) can only be controlled by TTL and versioned URLs. A content-hashed URL served with Cache-Control: max-age=31536000, immutable never needs invalidating, and the immutable flag stops browsers revalidating it on reload (RFC 8246). Across regions, invalidations must reach every regional cache, usually by replicating the database and running the log-tailing invalidator in each region, so a region never invalidates before its replica has the new data.

Down the stack

Layer 1The contract

A cache layer should state its staleness bound: "a read may return a value up to N seconds older than the latest commit", or "reads after an invalidation completes see the new value". TTL gives the first form with N equal to the TTL. Log-driven invalidation gives N equal to pipeline lag, typically under a second, with a guarantee that every commit eventually invalidates. Without leases or versions, neither form holds strictly, because the stale-set race can extend staleness to a full TTL.

Layer 2The mechanism

Memcached as deployed at Facebook issues a 64-bit lease token on a miss (Facebook NSDI 2013); a delete invalidates any outstanding lease for that key, and a set carrying an invalidated token is refused. Their invalidation daemon, mcsqueal, tails the MySQL commit log, extracts keys from annotated statements, batches deletes and sends them to routers in each region. Debezium does the same against PostgreSQL logical decoding or the MySQL binlog, publishing change events to Kafka for an invalidator service to consume. CDNs support surrogate keys (cache tags): responses are labelled with keys like product-42, and one purge call removes every cached URL carrying that tag. A soft purge marks matching objects stale instead of deleting them, so the next request is served the old copy while the CDN refetches (Fastly docs). Redis has no lease primitive, so teams implement compare-on-version sets with a Lua script, or with WATCH and MULTI, where the transaction aborts if a watched key changed (Azure caching guidance).

Layer 3The machine

A memcached or Redis delete is a single small request, roughly 100 to 300 microseconds round trip inside a data centre, so invalidation itself is cheap; the cost is in the misses it causes. Log tailing reads the binlog sequentially from disk or page cache, and its lag is dominated by batching intervals and cross-region replication, typically 10 ms to a few seconds. A CDN purge must reach hundreds of edge locations. Cloudflare reported a global median of 149 ms for tag, hostname and prefix purges in August 2024, down from about 1.6 seconds in May 2022 (Cloudflare); slower providers take seconds to minutes. Memory matters too: a cache with tombstones or version entries stores extra bytes per key, and at 500 million keys an extra 16 bytes each is 8 GB.

Why it's built this way

  1. Databases are expensive to read at high rates, so we copy hot data into memory close to readers.
  2. Copies drift when the source changes, and pushing every change to every copy is costly, so TTL bounds drift cheaply.
  3. Tighter freshness needs explicit invalidation, and concurrent readers and writers race, so caches add leases or versions.
  4. Application code forgets or crashes before invalidating, so the commit log, which records every change in order, becomes the invalidation source.
  5. Outer caches (browsers, CDNs) cannot be reached reliably, so immutable versioned URLs replace invalidation there.

Diagram

sequenceDiagram
    participant R as "Reader"
    participant C as "Cache"
    participant DB as "Database"
    participant W as "Writer"
    R->>C: get product:42
    C-->>R: miss, lease token L1
    R->>DB: SELECT (reads v1)
    W->>DB: UPDATE commit v2
    W->>C: delete product:42 (voids L1)
    R->>C: set v1 with L1
    C-->>R: rejected, lease invalid
    R->>C: next get misses, reads v2

Worked example

A product catalog serves 50,000 reads per second with a 98% hit rate from a Redis cluster; the database handles the 1,000 misses per second. Merchants make 200 price updates per second.

With TTL only at 10 minutes, a price change can be wrong for 10 minutes, which breaks checkout when the cart price differs from the charged price. With cache-aside deletes, staleness drops to milliseconds, but consider the stale-set race. A reader's miss-to-set window is about 5 ms (database read plus serialisation). Each update deletes one key; the race needs a concurrent miss on that same key inside the window. For popular products, with 50 reads per second on one key at 2% miss probability, the expected misses in a 5 ms window are 50 x 0.02 x 0.005 = 0.005 per update. Across 200 updates per second that is about one stale entry per second, or 86,000 per day, each lasting up to the TTL. Leases bring that to near zero.

The team adds log-based invalidation via change data capture so batch jobs that update prices directly in SQL also invalidate. Pipeline lag is about 300 ms p99, so for checkout the service reads the price from the primary database rather than the cache, a deliberate exception covering 2% of traffic.

In the wild

Trade-offs

OptionStrengthsWeaknessesChoose when
TTL onlyNo coordination, simpleStale up to TTL; misses on unchanged dataStaleness of minutes is acceptable
Delete on write (cache-aside)Fresh within millisecondsStale-set race, every writer must rememberMost read-heavy app data
Write-through updateCache always populatedOut-of-order updates, writes to cold keysSmall hot datasets with one writer
Log-driven invalidation (CDC)Catches every commit, orderedPipeline to operate, lagMany write paths, strict correctness
Versioned keys or URLsNo invalidation neededOld versions linger until evictedStatic assets, immutable snapshots

Pitfalls in production

Leadership lens

Check yourself

  1. Why is deleting a key safer than updating it on write?

    Show answer

    Concurrent updates can reach the cache in a different order from their database commits and leave an old value. A delete makes the next reader fetch the committed value.

  2. How can a reader put a stale value into the cache after a writer deleted it?

    Show answer

    The reader fetched the old row before the write, then set it after the writer's delete; nothing stops the late set without leases or versions.

  3. Why drive invalidation from the database log rather than application code?

    Show answer

    The log records every committed change in order, including from batch jobs and tools, and survives application crashes between commit and delete.

Sources and further study

Primary

Systems docs

Books

Other curricula

Watch

Free resources

Open in the interactive app: practice, quiz, mock interview →