Stop Treating Your Cache Like Storage: Hard Lessons from Real Deployments

Your cache is not a database. The moment you start treating it like one, you're signing up for stale data, mysterious 504s, and a 3 a.m. page. I've seen teams burn weeks trying to make Redis durable, only to watch it fall over under load. The fix isn't a better cache—it's admitting what a cache is for.

Should I use Redis or Memcached for my cache?

Pick Redis unless you have a specific reason not to. Memcached is multithreaded and can use multiple cores, but it lacks advanced data structures (AWS). Redis gives you lists, sets, sorted sets, hashes, bit arrays, and hyperloglogs, plus snapshots, replication, transactions, pub/sub, and Lua scripting. That flexibility matters when your cache needs to do more than store strings—like maintaining a leaderboard with sorted sets or running atomic counters with INCR.

But here's the trap: Redis's extra features tempt you to use it as a primary data store. Don't. A cache-aside pattern—check the cache, fetch from the database on a miss, write the result back—is the most common approach for a reason (AWS). It keeps the database as the source of truth and the cache as a disposable accelerator.

What's the most common caching myth?

That adding a cache automatically makes everything faster. Wrong. A cache with a low hit rate is just extra latency. You want hit ratios in the 80% to 95% range for CDNs, and Redis typically delivers 90% to 95% hit rates with 1 to 5 ms latency (AWS). If your hit rate is 40%, you've added a network hop for nothing.

The other myth: that you can cache everything. You can't. Static content—files identical every time they're delivered—is easy. Dynamic content needs careful cache keys. The Vary header tells caches which request headers influenced the response, so responses get cached separately per listed header (MDN). Get that wrong and you'll serve German pages to English users.

How do I handle cache invalidation without losing my mind?

You don't. You design around it. RFC 9111 says unsafe methods like PUT, POST, or DELETE must invalidate the stored target URI when they return a non-error status. That's the standard, but it only covers the cache that sees the request. Your CDN might not.

For CDNs, use purge-by-single-file. Cloudflare's purge instantly removes a cached resource across all data centers, so the next request gets the latest version (Cloudflare). Pair that with short TTLs for anything volatile. Cloudflare caches 200/206/301 for 120 minutes by default when no cache headers are present, 302/303 for 20 minutes, and 404/410 for 3 minutes (Cloudflare). Those defaults are fine for static assets, dangerous for API responses.

What load balancing algorithm should I actually use?

Round robin for short, stateless connections. Least connections or least response time for long connections. Hashing or cookies for session persistence (system-design-primer). That's the guidance, and it holds up.

But the real lesson from case studies is that the algorithm matters less than health checks. nginx's max_fails directive defaults to 1, meaning one failed attempt during fail_timeout marks a server as failed (Nginx). That's aggressive. If your health check hits a transient blip, you'll eject a healthy server. Set max_fails to 0 to disable health checks only if you have another mechanism—otherwise tune it.

HAProxy gives you finer control: inter defaults to 2000 ms, fall to 3, and rise to 2 (HAProxy). That means three consecutive failures to mark dead, two successes to mark alive. For a database backend with long-lived connections, leastconn is the right call because it handles uneven session lengths better than round robin.

How do I pick between a CDN, a reverse proxy cache, and an application cache?

You don't pick. You layer them. A typical architecture goes browser cache, CDN edge, load balancer/proxy cache, application cache, distributed cache, then the database (AWS). Each layer catches what the previous one missed.

Here's the comparison that matters:

nginx's proxy_cache_path lets you set a file-based cache with a directory hierarchy and a keys_zone. The inactive parameter removes cached data not accessed within a given time—default 10 minutes—regardless of freshness (Nginx). That's a quiet killer: a rarely accessed but still-valid asset gets evicted. If you have long-tail content, bump that up.

What I'd actually do

Stop trying to make your cache perfect. Make it disposable. Run Redis with allkeys-lru eviction—the default is LRU, and allkeys-lru is recommended because a cache is not storage (AWS). Set maxmemory and let it evict. Don't enable persistence unless you have a specific recovery requirement; if you do, use RDB snapshots with save points like 'save 60 1000' (Redis).

Put a CDN in front of everything static. Use Cloudflare's tiered cache if you're on any plan—the default Smart Tiered Cache is available on Free, Pro, Business, and Enterprise (Cloudflare). It reduces origin load by having only upper-tier data centers contact the origin.

For load balancing, start with nginx round robin. It's the default and it works for most HTTP traffic. Add least_conn when your requests have wildly different durations. Use consistent hashing with the hash directive if you're caching on the load balancer and want to survive server restarts without blowing away your cache—the consistent parameter remaps only a few keys when servers change (Nginx).

Finally, measure your hit rate. If it's below 80%, your cache is costing you more than it saves. Fix the keys, not the tool.

Sources

• AWS (caching overview) - https://aws.amazon.com/caching/
• MDN (Vary) - https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Vary
• RFC 9111 (HTTP Caching) - https://httpwg.org/specs/rfc9111.html
• Cloudflare (purge by single-file) - https://developers.cloudflare.com/cache/how-to/purge-cache/purge-by-single-file/
• Nginx (HTTP load balancing) - https://nginx.org/en/docs/http/load_balancing.html
• Redis (persistence) - https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/