Cache Like a Pro: A Field Guide for the Overworked Architect

Imagine you're the engineer on call for an e-commerce site that's about to get slammed by a flash sale. The product page is dynamic, the database is already sweating, and your boss is asking why the site was slow last Black Friday. You've heard about caching, but you're not sure where to start. This is the moment to stop guessing and start layering.

The Layered Reality

You don't cache once; you cache everywhere. A typical architecture stacks caches from the user's browser all the way down to the database (AWS). Each layer catches a different type of request, and if you miss one, you're leaving performance on the table. Start with the browser, then a CDN, then your load balancer or reverse proxy, then an application cache, then a distributed cache like Redis or Memcached, and finally the DB itself. The goal is to make the database the last resort, not the first stop.

HTTP Headers: Your First Line of Defense

Before you write a single line of cache code, get your HTTP headers right. The Cache-Control header is your primary tool, and it's packed with directives: max-age, s-maxage, no-cache, no-store, private, public, and more (MDN). For a public, static asset like a logo, you might set Cache-Control: public, max-age=604800 to keep it fresh for a week. For a user-specific page, you'd use private to keep it out of shared caches. And if you're dealing with a resource that changes but you want to avoid a full round-trip, use validators like ETag or Last-Modified to enable conditional requests (MDN). The ETag is a version identifier; when the content hasn't changed, the server can return a 304 Not Modified, saving bandwidth (MDN).

CDN: Move the Cache Closer

A CDN caches your content on servers spread across the globe, so a user in Tokyo doesn't have to wait for a round-trip to your origin in Virginia (Cloudflare). Cloudflare's network spans 348 cities across 8 regions, which is a lot of edge nodes (Cloudflare). By default, Cloudflare caches certain status codes for a set time when no cache headers are present: 200/206/301 for 120 minutes, 302/303 for 20 minutes, and 404/410 for 3 minutes (Cloudflare). That's a nice safety net, but you should set your own headers for precise control. A CDN can absorb 70% or more of origin requests, with hit ratios often reaching 80% to 95% (AWS). That's a huge load reduction. For dynamic content that changes per user, you might not want to cache at all, but for static assets, a CDN is a no-brainer.

Redis: The Distributed Cache Workhorse

When your app needs to cache data that's shared across servers, Redis is a common choice. It's an in-memory key-value store with typical hit rates of 90% to 95% and 1 to 5 ms latency, and adding it can reduce database CPU by 70% to 90% (AWS). You'll likely use the cache-aside pattern: check the cache first, on a miss fetch from the database, and write the result back (AWS). But don't treat Redis as a permanent store. Use eviction policies like allkeys-lru (least recently used) because a cache is not storage (AWS). Redis offers a variety of eviction policies, from noeviction (which errors when full) to volatile-ttl (evicts keys with the shortest TTL) (Redis). For a simple cache, allkeys-lru is often the right call. If you need to store more complex data, Redis supports lists, sets, sorted sets, hashes, streams, and more (Redis). And if you need atomicity, you can use Lua scripting via EVAL, which runs atomically (Redis).

Cache Invalidation: The Hard Part

You can't just set and forget. When data changes, you must invalidate the cache. HTTP has a rule: when a PUT, POST, or DELETE request hits the origin and returns a non-error status, caches must invalidate the stored URI (RFC 9111). That's a baseline. For a CDN, you might need to purge the cache. Cloudflare lets you purge by single-file, which instantly removes a cached resource across all data centers (Cloudflare). But you don't want to purge everything on every write; that defeats the purpose. Instead, use versioned URLs or ETags to let clients revalidate. And consider stale-while-revalidate, an RFC 5861 extension that lets a cache serve a stale response while it fetches a fresh one in the background (RFC 5861). This is a lifesaver for slow-changing data; it keeps the user experience snappy while the cache updates.

Load Balancing: The Cache's Partner

Your cache is only as good as your load balancer's ability to route requests consistently. If you have multiple app servers and each has its own local cache, you need sticky sessions or a shared cache. A load balancer can use algorithms like round robin, least connections, or IP hash (system-design-primer). For a cache-heavy workload, you want to maximize hit ratio, so consider consistent hashing. Nginx supports the hash key consistent directive, which uses ketama consistent hashing so that adding or removing a server remaps only a few keys (Nginx). That's crucial for a distributed cache; without it, you'd get a thundering herd of cache misses on every scale event. HAProxy also offers various algorithms, including roundrobin and leastconn (HAProxy). Choose based on your traffic pattern: round robin for short, stateless connections, least connections for long sessions (system-design-primer).

Putting It All Together: A Concrete Example

Let's say you're serving a product catalog. You set Cache-Control: public, max-age=3600 on the product images, so the CDN and browsers cache them for an hour. For the product JSON API, you use ETag and Cache-Control: no-cache so clients always revalidate but can use a 304 to avoid downloading the full body. On the application side, you use Redis with allkeys-lru and a TTL of 15 minutes for product details. When a product is updated, you invalidate the Redis key and purge the CDN URL for that product. You also set stale-while-revalidate=300 on the API responses, so if the cache is stale, it can serve the old data while fetching fresh in the background. Your load balancer uses consistent hashing on the product ID so that all requests for a given product hit the same backend, which can use a local cache as well. This layered approach keeps your origin happy and your users even happier.

Sources

• RFC 9111 - https://httpwg.org/specs/rfc9111.html
• RFC 5861 - https://www.rfc-editor.org/rfc/rfc5861.txt
• MDN Cache-Control - https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control
• Cloudflare CDN - https://www.cloudflare.com/learning/cdn/what-is-a-cdn/
• Redis Eviction - https://redis.io/docs/latest/develop/reference/eviction/
• Nginx Load Balancing - https://nginx.org/en/docs/http/load_balancing.html