A Node.js cache stampede happens when many requests miss the same cached value at once and all ask the database to rebuild it. The cache was meant to reduce work, but its expiry creates a sudden burst of identical queries.
We can start with a small improvement: let simultaneous callers in one process share the same refresh. Then we’ll look at what changes when several processes are involved.
Separate the value from the refresh in progress
A normal cache stores a value and expiry time. Add another map for an in-flight refresh promise. The first caller creates it; later callers await that same promise.
Store the promise before awaiting the loader. Otherwise, several callers can all observe that no refresh exists and start their own. Remove it when the refresh settles so a failed load doesn’t poison the key forever.
Key the cache by tenant and all arguments that affect the result. Sharing a refresh for two different businesses is a data-isolation bug, not a useful change.
Decide whether old data can be served
Some data can tolerate stale-while-revalidate: return an older description while one background refresh updates it. Other data needs a fresh result or a clear failure, such as an current source stock allocation.
Define fresh and maximum-stale deadlines separately. “Serve stale” without an age limit can keep returning a record that has been wrong for days during an outage.
Keep negative results under their own policy. An item that wasn’t found a moment ago may have just been created, and a cached permission denial or allowance may need stronger invalidation.
Build single-flight refresh
The example launches twenty simultaneous local requests for one key. It verifies that the underlying loader runs once, then checks that a failed loader can be retried.
import assert from 'node:assert/strict';
import {setTimeout as sleep} from 'node:timers/promises';
const pending=new Map(),cache=new Map();
let loads=0;
function get(key,load){
if(cache.has(key))return Promise.resolve(cache.get(key));
if(pending.has(key))return pending.get(key);
const work=Promise.resolve().then(load).then(value=>{cache.set(key,value);return value;})
.finally(()=>pending.delete(key));
pending.set(key,work);return work;
}
const values=await Promise.all(Array.from({length:20},()=>get('tenant:a:catalogue',async()=>{loads++;await sleep(10);return 'catalogue';})));
assert.equal(loads,1);assert.equal(values.length,20);
await assert.rejects(get('broken',async()=>{throw new Error('Temporary');}));
assert.equal(await get('broken',async()=>'recovered'),'recovered');
assert.equal(pending.size,0);
console.log('PASS: twenty simultaneous callers share one refresh; failures can retry');
That is real concurrent asynchronous execution in one Node.js process. It doesn’t prove that twenty independent servers share one refresh. Each process has its own maps.
The example also avoids retaining rejected promises as permanent cache entries. That small failure path matters when the database is temporarily unavailable.
Coordinate across processes carefully
At larger scale, each process can still issue one refresh for the same key. A shared lease or a refresh service can reduce that duplication, but the lease needs ownership and expiry rules.
An expired lease doesn’t stop its old owner from finishing. If the refreshed value is versioned, reject an older write that arrives after a newer one. Use a token or compare-and-set rule appropriate to the storage system.
Redis documents lock ownership and validity considerations. Don’t turn a short lock example into a blanket distributed-correctness guarantee. Distributed lock reference.
Avoid synchronising expiry across the whole catalogue
If every entry expires at the same fixed boundary, different keys can still flood the source together. Add bounded variation to expiry where the product’s freshness rules allow it.
Also bound the number of concurrent refreshes for different keys. Single-flight prevents duplicates for one key; it doesn’t protect the database from thousands of distinct misses.
Prewarming can help known popular entries, but it should follow a budget. Refreshing every possible key on a schedule can cost more than serving actual demand.
Handle a cache outage as its own workload
If the shared cache fails, falling through directly to the database for every request can overload the database. Use a bounded fallback path, local short-lived values where acceptable, and explicit rejection or degradation when capacity is exhausted.
Measure the source query rate during the outage. A healthy cache hit ratio from before the failure says little about the fallback’s safety.
Don’t use stale personal permissions or balances simply to preserve availability. Choose the fallback according to the operation’s correctness requirements.
Keep refresh failures visible
Record refresh duration, loader failures, callers sharing each refresh and the age of served stale data. Avoid logging the full cache key if it contains personal identifiers; use a safe correlation strategy.
If the same entry refreshes continuously, inspect whether its TTL is too short, the loader never succeeds or invalidation is firing repeatedly. Increasing cache size won’t fix those problems.
Test both one key and many keys
Start with simultaneous misses for a hot key, as the example does. Then test many unique keys, a slow loader, a failed loader and a cache connection outage.
For a distributed lease, pause the first owner until its lease expires and let a second owner complete. Verify that the first cannot overwrite the newer value when it wakes up.
Start with in-flight promise sharing in the process you already run. It’s a small, testable improvement. Add cross-process coordination only when measurements show that the remaining duplicate refresh work matters.
Run the example and adapt it
Save the code as 22-cache-stampede-singleflight.mjs and run it with Node.js 24 or later:
node 22-cache-stampede-singleflight.mjs
The code above includes assertions you can run locally. It tests the local decision logic with controlled inputs; it doesn’t connect to a live provider or establish production performance. Keep that boundary when adapting it to your application.
