Why Your Cache Hit Ratio Is a Lie (and How to Fix It)

You think your cache is working because your hit ratio is 80%. That's the lie. An 80% hit ratio sounds great—until you realize that the 20% of misses are your most expensive, most popular, or most dynamic requests. Hit ratio is a vanity metric if you don't know which requests are missing. You need to stop chasing a number and start chasing the right misses.

What Your Hit Ratio Isn't Telling You

Let's get blunt: a cache hit ratio is an average, and averages hide the worst offenders. You might have a 95% hit ratio on static images, but your API endpoints—the ones that hit your database on every call—might be at 0%. The average looks fine, but your database is still drowning. (AWS notes that typical cache hit rates for Redis are 90-95%, but that's for a well-configured cache, not a blanket promise.) So the first question isn't "What's my hit ratio?" but "What are the misses?" You need to instrument your cache to log cache keys and response times for misses. If you don't know your misses, you're flying blind.

Why Your Headers Are Making You Miss

One of the most common reasons for avoidable misses is bad cache headers. You might be setting Cache-Control: no-cache on a resource that could be cached for hours, or you might be missing the Vary header and caching the wrong variant. Here's the fix: use max-age for static assets, but set it relative to the origin, not the client. (MDN notes that max-age is seconds after the response is generated on the origin, not after it's received.) And for anything that varies by user, language, or device, you must set Vary correctly. If you don't, you'll serve the wrong cached version—or worse, you'll get cache poisoning. (MDN: Vary lists the request headers that influenced the response, so each variant is cached separately.)

The CDN Is Not a Magic Bullet

You might think adding a CDN will fix everything. It won't, unless you configure it. A CDN like Cloudflare caches static content by default, but dynamic content often isn't cached unless you tell it to. (Cloudflare notes that static content is identical every time, and browsers and CDNs can cache it for a set period.) For dynamic content, you can use stale-while-revalidate to serve stale content while fetching fresh in the background. (RFC 5861 defines this extension.) But even with a CDN, you need to think about the cache key: if you don't include the right query parameters or headers in the cache key, you'll get misses. And if you're using a CDN, you need to think about cache invalidation: purging by single-file is instant, but if you have many files, you might need a smarter strategy. (Cloudflare's single-file purge removes the asset across all data centers.)

When Your Cache Is Too Smart for Its Own Good

Now, let's talk about the application-level cache. You're using Redis, and you've set maxmemory-policy to noeviction because you're afraid of losing data. That's a mistake. A cache is not storage; it's a performance layer. If you set noeviction, Redis will start returning errors when it's full, and your app will fail. The right policy is allkeys-lru or allkeys-lfu to evict the least recently or least frequently used keys. (AWS recommends allkeys-lru because a cache is not storage.) And if you're using Redis for rate limiting or counters, don't use SETEX for everything—use INCR with an expiration to keep it simple. (Redis INCR is O(1) and creates the key at 0 if it doesn't exist.)

The Real Fix: Start with the Misses

Here's the blunt advice: Stop optimizing your hit ratio. Start optimizing your misses. Instrument your cache to log the keys that miss and the time it takes to fetch from the database. You'll likely find that a few endpoints are responsible for most of the cost. For those, you can do several things. First, ensure your cache headers are correct so that CDN and browser caches take some load off. Second, for dynamic data, use a cache-aside pattern: check the cache first, fetch on a miss, then write back. (AWS describes this as the most common pattern.) Third, if you have a stampede problem, use proxy_cache_lock in nginx to prevent multiple requests from populating the same cache key at once. (Nginx proxy_cache_lock lets only one request populate, and others wait up to a timeout.) And fourth, consider using stale-while-revalidate to serve stale data while refreshing in the background, so your users never wait for a database call.

The single most important thing to remember: Your cache hit ratio is a symptom, not a goal. Focus on the misses that matter, and the ratio will take care of itself.

Sources

• MDN (Cache-Control) - https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control
• MDN (Vary) - https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Vary
• RFC 5861 (stale-while-revalidate) - https://www.rfc-editor.org/rfc/rfc5861.txt
• Cloudflare (purge by single-file) - https://developers.cloudflare.com/cache/how-to/purge-cache/purge-by-single-file/
• Redis (eviction) - https://redis.io/docs/latest/develop/reference/eviction/
• Nginx (proxy module) - https://nginx.org/en/docs/http/ngx_http_proxy_module.html
• AWS (caching overview) - https://aws.amazon.com/caching/