Question

Why is cache invalidation considered hard?

Vault Verified
Curated Intelligence
Definitive Source
Answer

The famous quip from Phil Karlton is that there are two hard things in computer science: cache invalidation and naming things. The joke endures because the problem is genuinely difficult, and the difficulty is structural rather than incidental.

A cache is a second copy of data. The moment you have two copies, they can disagree, and the whole problem is knowing when the copy has become wrong.

Why that is hard:

The cache does not know when the source changed. Something must tell it, and every mechanism for doing so has failure modes. Time-based expiry (TTL) is simple but always wrong in one direction — too short and you lose the benefit, too long and you serve stale data. Event-based invalidation is precise until an event is missed, dropped or arrives out of order.

Dependencies are not obvious. A cached page may depend on a dozen records. Changing one should invalidate it, but nothing in the system records that relationship unless you build it deliberately, and keeping that map accurate as code evolves is where it decays.

Distributed caches multiply it. With many cache nodes, invalidation must reach all of them, and between the update and the last node receiving it, different users see different data.

The thundering herd. Expire a popular entry and every waiting request stampedes the origin simultaneously — sometimes taking it down. Mitigations include request coalescing, staggered TTLs and serving stale while revalidating.

Correctness is invisible. A missed invalidation does not crash anything. It quietly serves wrong data, often for a long time, and is typically discovered by a confused user rather than monitoring.

Common strategies: short TTLs where staleness is tolerable; cache keys that embed a version or content hash, so changed content simply has a new key and old entries expire harmlessly; and write-through caches updating on write.

The most reliable advice: cache less than you think, and make staleness visible.

Related Questions