Skip to main content

Caching is Easy Until It Isn't

Md. Navid Razzaque
Md. Navid Razzaque
Published 11 November 2026·8 min read
Share Article:
Caching is Easy Until It Isn't

The day your cache saves you is the day you stop thinking about it. That's exactly when it starts planning its revenge.

I've seen it happen more times than I'd like to admit. A service is struggling under load. Queries are slow, users are complaining, the on-call engineer is sweating. Someone adds a cache. Latency drops. The dashboard goes green. Everyone breathes again.

And then, three months later, on a Tuesday morning when traffic spikes - maybe a sale, maybe a feature launch, maybe nothing at all - the database gets absolutely obliterated. The cache is sitting right there, full of data, technically working. And somehow the database is on its knees.

The cache didn't fail. The team just never learned what the cache was quietly assuming on their behalf. This is that story.

You don't add a cache. You add a set of assumptions.#

When you put a cache in front of your database, you're not just adding speed. You're making a deal with the universe:

"I accept that some readers will see slightly old data, in exchange for not hammering my database on every read."

That's a reasonable deal. Most of the time it's the right one. But the fine print is long, and most teams never read it.

The cache doesn't know your business logic. It doesn't know which data can be stale for ten seconds and which data being stale for one second means someone gets charged twice. It just stores and retrieves, blissfully unaware of what's actually at stake.

The assumptions are yours. And each one of them has a failure mode.

The stampede you didn't see coming#

Here's a scenario. Your homepage shows a featured product list. You cache it for 60 seconds because it barely changes. Life is good.

Then at 9:03am - maybe you just sent a push notification to a million users, maybe traffic just naturally spikes - 800 concurrent requests arrive at your service. They all check the cache. The key expired at 9:02:59. Every single one of them gets a cache miss. Every single one of them goes to the database. Every single one of them runs the same expensive query, gets the same result, and tries to write it back to the cache.

Your database just absorbed 800 identical queries in the same second for no reason. This is called a cache stampede, and it's one of the more spectacular ways a cache can make things worse than if it didn't exist.

The fix isn't complicated once you know the problem. The simplest is a mutex on cache miss - only one request gets to recompute the value, the rest wait. A more elegant option is probabilistic early expiration - each request has a small chance of recomputing slightly before expiry, so the cache refreshes gradually without a cliff edge. The most production-friendly is background refresh - a separate job proactively refreshes hot keys before they expire. Your handlers never see a miss at all.

The avalanche nobody warned you about#

A stampede is one key expiring at a bad time. An avalanche is all your keys expiring at the same time.

This happens right after a deployment or service restart that wiped the cache. You repopulate it - and since all the keys were written in the same 30-second window, they all have the same TTL. They all expire together. What you experience looks exactly like a database outage even though the database is perfectly healthy.

The fix is almost embarrassingly simple: jitter your TTLs.

One line of code. Genuinely saves databases.

The attack vector nobody mentions in caching tutorials#

Everything we've talked about so far happens by accident - bad timing, missing jitter, an unlucky deployment. This one can be deliberate.

Imagine someone runs a script. It hits your API thousands of times per second with keys that don't exist - /user/9999999999, /product/-1, /order/randomgarbage. Your cache checks for each key. Nothing there. Falls through to the database. The database checks. Also nothing there. Returns empty.

The cache offered zero protection. Every single request landed on your database. Do that at enough volume and your database goes down - not because the data is being stolen, but because the lookup cost of confirming "this doesn't exist" adds up fast enough to saturate your DB completely. This is called cache penetration, and it's one of the cleaner ways to take down a service that looks well-protected on the surface.

The quick fix - cache the nothing

When your DB returns empty for a key, write a null marker back to the cache with a short TTL. Next time that same key arrives, the cache hits, returns null immediately, DB is never touched.

The tradeoff is minor: if a record gets created within the 30-second window, you might serve stale null briefly. Short TTL keeps that window tight.

The proper fix at scale - Bloom filters

Caching null results works well for repeated keys. But a sophisticated attacker rotates keys constantly - every request is a different fake ID. Null caching doesn't help because you've never seen the key before.

A Bloom filter is a data structure that lives in memory and answers one question extremely fast: "Has this key ever existed in the system?" You populate it with all valid keys. Before a request even touches your cache or database, you check the Bloom filter. If it says the key has never existed, you reject the request right there. No cache lookup. No DB query. Just gone.

A Bloom filter never produces false negatives. If it says a key doesn't exist, that's definitive. Redis has a Bloom filter module built in.

The elegant property: a Bloom filter can occasionally say a key might exist when it doesn't - a false positive - but that just means an unnecessary cache lookup, not a DB hit. For any system where keys follow a predictable namespace, this is the cleanest defence at scale.· · ·

Stale-while-revalidate: the pattern you probably want but haven't named#

Here's an honest question: when your cache key expires, what do you actually want to happen?

You probably don't want the user to wait while you fetch fresh data from the database. You probably don't want the database to get hit by everyone who was waiting. What you actually want is: serve the old value right now, and go get a fresh one in the background.

That's stale-while-revalidate. When a key "expires", you don't delete it - you mark it as stale but keep it around. Any request that hits a stale key gets the old value immediately while a background refresh fetches the fresh one. The next request gets the updated value, but nobody waited for it.

The honest tradeoff: for a brief window, some users see data that's slightly out of date. For most things - product listings, user profiles, recommendation feeds - this is completely fine. For account balances or anything where "right now" actually matters, it isn't. Know which category your data falls into before you decide.

What not to cache (and why people cache it anyway)#

At some point caching becomes reflexive. Something is slow, the instinct is to cache it. But not everything belongs in a cache.

Permissions and access control - if you cache whether a user can access a resource, you need to be very precise about when that cache invalidates. Someone gets their permissions revoked. Your cache still says they have access. For the next five minutes, they do. This is the kind of bug that shows up in a security audit, not in your test suite.

Things that are slow because of a missing index - a query takes 800ms, someone adds a cache, the dashboard looks great. Two years later the cache expires at an inconvenient moment and everyone has forgotten why it was there. The cache fixed the symptom. The index would have fixed the problem.

Tiny, cheap things - there's overhead to a cache lookup: serialisation, network round trip to Redis, deserialisation. If what you're caching takes 2ms to compute and the cache lookup takes 1ms, you've saved 1ms and added failure surface area, a network dependency, and memory pressure.

So should you cache? Yes. But…#

None of this means don't cache. Caching is one of the most effective tools in backend engineering and most systems genuinely need it.

But the engineers who use it well think about it differently. They don't add a cache and move on. They ask:

What happens when this cache key expires at the worst possible moment?What happens when the cache goes down entirely?What happens if someone deliberately fires keys that don't exist?Is this data safe to serve stale, and for how long?Am I caching this because it's genuinely expensive, or because I haven't found the real problem yet?

Those questions don't take long to ask. But asking them before you deploy is a lot more comfortable than answering them at 2am when the database is down and the cache is full and nobody can explain why traffic keeps bypassing it anyway.

The cache is not magic. It's a deal.

Found this useful? Share it with your peers:

More Blogs & Related Reads

Continue reading technical insights written by Exabyting engineers