Stop Caching Everything: The Case for Deliberate Cache Layers

Imagine you're the engineer on call at 2 a.m. Your origin server is melting under a traffic spike that your load balancer can't absorb. You scramble to add a cache layer, but it's too late—the database is already down. This scenario plays out more often than it should because teams treat caching as an afterthought, a single knob to turn when things go wrong. We've been there, and we've learned that the only way to build a system that survives is to design caching deliberately, layer by layer, with a clear purpose for each one.

Here's our thesis: you should not cache everything. Instead, you should decide what to cache at each layer—browser, CDN, proxy, application, and distributed cache—and be ruthless about what belongs where. A blanket cache-all approach is a recipe for stale data, wasted memory, and a false sense of security.

Layers Are Not Optional

The classic architecture isn't a suggestion; it's a map of what works. From the user outward, you have the browser cache, then a CDN edge, then a load balancer or proxy cache, then an application cache, then a distributed cache like Redis or Memcached, and finally the database (AWS). Each layer has a different job, and mixing them up is a mistake. For example, you wouldn't use a CDN to cache a user's personalized dashboard—that's a job for an application or distributed cache. The CDN is for static assets and content that's identical for everyone.

Consider static content: files that are the same every time they're served. Browsers and CDNs can cache these for a set period and keep serving them while they're requested (Cloudflare). That's a no-brainer. But dynamic content, like a shopping cart or a news feed, needs a different approach. You might use a distributed cache with a short TTL, or you might not cache it at all if it changes too frequently.

Pick Your Poison: Eviction and TTL

Once you decide what to cache, you must decide how long to keep it. HTTP headers give you fine control: Cache-Control: max-age=604800 keeps a response fresh for 7 days (MDN). But freshness is only half the story. What happens when the cache is full? Redis, for example, offers a suite of eviction policies, but the default is noeviction, which returns an error when you try to write more data than memory allows (Redis). That's a trap. We recommend allkeys-lru because a cache is not storage—it should evict the least recently used items to make room for new ones (AWS).

And don't forget about invalidation. HTTP caching has a rule: when an unsafe request like POST, PUT, or DELETE hits the origin, caches must invalidate the stored URI (RFC 9111). But that's not enough for many applications. If you need to purge a single file across a CDN, you can do that instantly (Cloudflare). But if you're using a distributed cache, you might need to actively delete keys or use a versioning scheme. The point is: invalidation is not an afterthought; it's a design decision.

The Cache-aside Pattern and Its Limits

The most common pattern is cache-aside, also known as lazy loading: check the cache first, on a miss fetch from the database, then write back to the cache (AWS). It's simple and works well for many read-heavy workloads. But it has a weakness: a sudden spike in cache misses can hammer the database. That's where a CDN or proxy cache can help by absorbing some of that load. CDNs can absorb 70% or more of origin requests, and cache hit ratios commonly reach 80% to 95% (AWS). Those are big numbers, but they don't happen by accident.

For example, if you have a news website, you might cache the homepage for 5 minutes at the CDN, but the comments section might be dynamic and uncached. A CDN default might cache a 200 response for 120 minutes if no cache-control headers are present (Cloudflare). That's a long time for a breaking story. You need to set explicit TTLs based on how often content changes.

The Counter-argument: Just Cache Everything with Short TTLs

Some argue that you can bypass the complexity by caching everything with a short TTL, say 30 seconds. That way, you get some benefit without thinking about invalidation. We reject this for two reasons. First, short TTLs mean you're constantly revalidating, which can reduce the cache hit rate and put more load on the origin. Second, some data changes so frequently that even a 30-second stale window is unacceptable, like a stock price ticker. You need finer control.

Instead, we advocate for a layered approach where each cache has a specific role. Use the browser cache for static assets with long TTLs. Use a CDN for shared static content. Use a reverse proxy like nginx with its proxy_cache_path directive to cache dynamic responses at the edge (Nginx). Use Redis for frequently accessed database queries, with an eviction policy that fits your access pattern. And use HTTP validators like ETags and Last-Modified to avoid transferring unchanged content (MDN).

Here's a quick tip: if you're using Redis for caching, consider using SETEX to set a key with a TTL in one atomic command (Redis). That avoids the race condition of setting a key and then expiring it separately.

• Use round robin for short, stateless connections; least connections for long ones; and hashing for session persistence (system-design-primer).
• Redis Cluster uses 16384 hash slots, and a three-node cluster might split them as 0-5500, 5501-11000, and 11001-16383 (Redis).
• nginx worker_processes auto sets it to the number of CPU cores, a good starting point (Nginx).

Warning: Do not enable Redis persistence for a cache. You don't want to replay every write on restart; you want a fresh start.

Sources

• AWS (caching overview) - https://aws.amazon.com/caching/
• Cloudflare (default cache behavior) - https://developers.cloudflare.com/cache/concepts/default-cache-behavior/
• Redis (eviction) - https://redis.io/docs/latest/develop/reference/eviction/
• RFC 9111 (HTTP Caching) - https://httpwg.org/specs/rfc9111.html
• Nginx (proxy module) - https://nginx.org/en/docs/http/ngx_http_proxy_module.html

Takeaway: Caching is not a single switch; it's a series of deliberate choices. Start with the data that benefits most, set explicit TTLs, and choose eviction policies that match your access patterns. The goal is to keep the origin from being a bottleneck, not to make every byte cacheable. If you do it right, you'll sleep better at 2 a.m.