Level 3 · Architecture

Caching, and what it really costs

A cache turns a latency problem into a correctness problem. Every layer you add buys speed and money and pays for it in staleness, and the hit rate decides whether the trade was worth making.