Preventing Cache Stampedes When Many Requests Miss the Same Key

Caching helps applications respond faster by keeping frequently requested data in memory or in a distributed cache such as Redis. Instead of querying a database or calling an external API for every request, the application can reuse a previously calculated result.

However, caching introduces a problem that often appears under heavy traffic. When a popular cache entry expires, many requests may attempt to rebuild the same value at nearly the same time. Each request discovers that the value is missing and independently queries the database or calls the downstream service.

This situation is known as a cache stampede.

A cache stampede can overload a database, increase API response times, and cause cascading failures across services. Even an application with a high cache hit rate can experience this problem when a frequently accessed key expires.

In this article, we will explore why cache stampedes happen and how to prevent them in C# applications using IMemoryCache, HybridCache, and distributed coordination techniques.

What Causes a Cache Stampede?

Consider an API that returns product information. The application stores the result in a cache for five minutes.

During those five minutes, thousands of requests can retrieve the cached value without querying the database. When the entry expires, the next requests find a cache miss.

If 200 requests arrive at approximately the same time, all 200 may query the database before any request finishes rebuilding the cache entry.

The sequence looks like this:

Popular cache entry expires

Product information is no longer available in the cache.

Multiple requests arrive

Each request observes the same cache miss.

Multiple database queries execute

Query 1
Query 2
Query 3 ... N

Database receives unnecessary load

Identical work runs repeatedly instead of once.

The problem is not simply that the cache entry expired. The problem is that the application allows multiple requests to perform the same expensive work concurrently.

If the database slows down under this load, requests take longer to finish. More requests remain in flight, which can increase resource consumption and make the outage worse.

Why a Normal Cache Lookup Is Not Enough

A common implementation uses IMemoryCache to cache the result of an expensive database query.

public async Task<Product?> GetProductAsync(
    int productId,
    CancellationToken cancellationToken)
{
    string key = $"product:{productId}";

    if (_cache.TryGetValue(key, out Product? product))
    {
        return product;
    }

    product = await _dbContext.Products
        .AsNoTracking()
        .SingleOrDefaultAsync(
            p => p.Id == productId,
            cancellationToken);

    if (product is not null)
    {
        _cache.Set(
            key,
            product,
            TimeSpan.FromMinutes(5));
    }

    return product;
}

This implementation is reasonable for basic caching, but it has a race condition.

Imagine that three requests execute TryGetValue before the first database query completes. All three receive a cache miss. Each request starts its own query, and all three eventually write the same key.

The cache prevents repeated work after the value is populated, but it does not prevent duplicate work while the value is being created.

A safer design needs to coordinate cache population, not just cache access.

Use Single-Flight Loading with a Semaphore

One approach is to allow only one request at a time to rebuild a particular cache entry within the same application process.

The first request acquires a semaphore and loads the data. Other requests wait. Once the first request populates the cache, waiting requests check the cache again and reuse the result.

using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Caching.Memory;
using System.Collections.Concurrent;

public sealed class ProductCacheService
{
    private readonly IMemoryCache _cache;
    private readonly AppDbContext _dbContext;

    private static readonly ConcurrentDictionary<int, SemaphoreSlim>
        Locks = new();

    public ProductCacheService(
        IMemoryCache cache,
        AppDbContext dbContext)
    {
        _cache = cache;
        _dbContext = dbContext;
    }

    public async Task<Product?> GetProductAsync(
        int productId,
        CancellationToken cancellationToken)
    {
        string key = $"product:{productId}";

        if (_cache.TryGetValue(key, out Product? cached))
        {
            return cached;
        }

        SemaphoreSlim semaphore = Locks.GetOrAdd(
            productId,
            _ => new SemaphoreSlim(1, 1));

        await semaphore.WaitAsync(cancellationToken);

        try
        {
            // Another request may have populated the cache
            // while this request was waiting.
            if (_cache.TryGetValue(key, out cached))
            {
                return cached;
            }

            Product? product = await _dbContext.Products
                .AsNoTracking()
                .SingleOrDefaultAsync(
                    p => p.Id == productId,
                    cancellationToken);

            if (product is not null)
            {
                _cache.Set(
                    key,
                    product,
                    TimeSpan.FromMinutes(5));
            }

            return product;
        }
        finally
        {
            semaphore.Release();
        }
    }
}

The second cache check is essential. Without it, each waiting request would query the database after acquiring the semaphore, even though the previous request had already populated the cache.

This example illustrates the single-flight pattern: multiple callers request the same value, but only one performs the expensive load.

There are two important production considerations.

First, the static dictionary retains semaphores for every distinct product ID accessed. If the key space grows without bound, this can cause a memory leak. Use a bounded lock registry, a well-tested keyed-lock implementation, or a cache-loading abstraction that handles per-key coordination safely.

Second, this solution coordinates requests only within one process. If an application runs on ten servers, each server can still initiate one database query for the same missing key. A distributed application needs a cross-instance strategy when local coordination is not sufficient.

Use HybridCache for Cache Stampede Protection

For modern .NET applications, HybridCache is another option worth considering. It provides a unified caching API that can use in-process caching and an optional distributed cache, and it includes protection against multiple concurrent callers performing the same cache fill within the applicable coordination scope.

Add the package if your application does not already reference it:

dotnet add package Microsoft.Extensions.Caching.Hybrid

using Microsoft.Extensions.Caching.Hybrid;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddHybridCache();

var app = builder.Build();

You can then use the cache-aside pattern through GetOrCreateAsync:

public sealed class ProductService
{
    private readonly HybridCache _cache;
    private readonly AppDbContext _dbContext;

    public ProductService(
        HybridCache cache,
        AppDbContext dbContext)
    {
        _cache = cache;
        _dbContext = dbContext;
    }

    public ValueTask<Product?> GetProductAsync(
        int productId,
        CancellationToken cancellationToken)
    {
        return _cache.GetOrCreateAsync(
            $"product:{productId}",
            async token =>
            {
                return await _dbContext.Products
                    .AsNoTracking()
                    .SingleOrDefaultAsync(
                        p => p.Id == productId,
                        token);
            },
            cancellationToken: cancellationToken);
    }
}

The factory performs the database query only when the cache needs a value. Concurrent callers for the same key can share the result rather than independently running the factory.

HybridCache is particularly useful when you want a supported caching abstraction without implementing your own per-key locking logic. However, do not assume that its coordination provides a global lock across every server instance. If the database cannot tolerate one cache fill per instance, you may need distributed coordination or a different caching architecture.

Also, avoid sharing a single EF Core DbContext across concurrent operations. DbContext is not thread-safe. Register it with an appropriate lifetime, commonly scoped to the request, and make sure the cache factory does not capture a context that is used concurrently by unrelated requests.

Serve Stale Data While Refreshing the Cache

Sometimes the best solution is to avoid making every request wait for a cache rebuild.

For data that can tolerate a short period of staleness, the application can keep the previous value available while refreshing it in the background. This approach is often called stale-while-revalidate.

For example, a product catalog may be allowed to display data that is one minute old while a background task refreshes the cache. Requests continue receiving the previous value rather than waiting for the database.

A practical design separates the fresh period from the longer retention period:

Cache state	Application behavior
Fresh value	Return the cached value
Recently expired value	Return the old value and trigger one refresh
No usable value	Load the value before responding
Refresh failure	Keep the previous value when policy permits and retry later

This strategy requires explicit freshness metadata. If the application simply removes the old entry when it expires, there is nothing to serve during the refresh.

It is also unsuitable for every type of data. Account balances, authorization decisions, payment status, and inventory availability may require stricter freshness guarantees. Stale data should be served only when the business rules allow it.

Add TTL Jitter to Avoid Synchronized Expiration

A cache stampede can affect many different keys at once. This often happens when an application loads a large set of records and gives every entry the same expiration time.

For example, if 50,000 product entries are cached for exactly ten minutes after a deployment, many of them may expire around the same time.

TTL jitter introduces a small random variation in the expiration period.

public static class CacheExpiration
{
    public static TimeSpan WithJitter(
        TimeSpan baseTtl,
        double jitterFraction = 0.2)
    {
        if (baseTtl <= TimeSpan.Zero)
        {
            throw new ArgumentOutOfRangeException(
                nameof(baseTtl));
        }

        if (jitterFraction < 0 || jitterFraction > 1)
        {
            throw new ArgumentOutOfRangeException(
                nameof(jitterFraction));
        }

        double factor = 1 + (
            (Random.Shared.NextDouble() * 2 - 1)
            * jitterFraction);

        return TimeSpan.FromMilliseconds(
            baseTtl.TotalMilliseconds * factor);
    }
}

You can use the helper when adding an entry:

_cache.Set(
    key,
    product,
    CacheExpiration.WithJitter(
        TimeSpan.FromMinutes(10)));

With a 20% variation, an entry configured for ten minutes receives a TTL between approximately eight and twelve minutes.

TTL jitter spreads expiration times across many keys. It does not guarantee that only one request loads a particular key, so it should complement single-flight loading rather than replace it.

Use a Distributed Lock Only When You Need One

For workloads where even one cache refresh per application instance is too expensive, a distributed lock can coordinate cache population across servers.

A distributed lock must handle lease expiration, ownership, and failures correctly. A worker can pause after acquiring a lock, lose the lease, and then resume while another worker is already refreshing the same value.

For that reason, avoid implementing a distributed lock using only a simple shared Boolean flag. Use a well-designed coordination mechanism with atomic acquisition and safe release. For critical writes, fencing tokens or equivalent storage-side protections may also be necessary.

Before adding distributed locking, measure the problem. A local single-flight mechanism plus TTL jitter may be enough for many workloads. Distributed coordination adds network requests, failure modes, and operational complexity.

Common Mistakes to Avoid

Mistake	Why it causes problems	Better approach
Using only a cache lookup	Concurrent misses can trigger duplicate queries	Coordinate cache population per key
Locking every cache key with one global lock	Unrelated requests block one another	Use per-key coordination
Forgetting the second cache check	Waiting requests may reload data unnecessarily	Check again after acquiring the lock
Using process-local locks across multiple servers	Each server can still refresh the same key	Consider distributed coordination when required
Giving every entry the same TTL	Many keys may expire together	Add TTL jitter
Serving stale data without a policy	Critical decisions may use outdated information	Define acceptable staleness per data type
Keeping an unbounded dictionary of locks	Lock objects can accumulate indefinitely	Use bounded or managed keyed coordination
Caching failures indefinitely	Temporary dependency failures can become persistent application errors	Apply a deliberate failure-caching and retry policy

Best Practices for Production Applications

Measure cache hit rates, database query volume, refresh duration, and concurrent misses.
Use per-key single-flight coordination for expensive cache fills.
Consider HybridCache when it fits the application's caching requirements.
Add TTL jitter when many entries can expire together.
Use stale-while-revalidate only for data that can safely be slightly outdated.
Keep cache keys specific to the tenant, user, locale, or other dimensions that affect the result.
Avoid caching sensitive information under keys that can be shared across users.
Set timeouts and cancellation policies for expensive database or API calls.
Bound the size of local caches and any auxiliary lock registry.
Test cold-cache behavior under concurrent load, not only steady-state cache hits.

Advantages and Limitations

Advantages

Single-flight loading reduces duplicate work for the same cache key.
TTL jitter spreads cache expiration across different times.
Stale-while-revalidate can keep response times low during refreshes.
Hybrid caching can simplify cache-aside implementations.
Distributed coordination can reduce redundant refreshes across multiple servers.

Limitations

Local coordination does not guarantee one refresh across an entire cluster.
Distributed locks introduce additional failure modes and latency.
Serving stale values may violate freshness requirements.
Cache-loading abstractions do not remove the need for correct database and dependency lifetimes.
TTL jitter reduces synchronized expiration but does not eliminate concurrent misses for an individual key.

Summary

A cache stampede happens when multiple requests attempt to rebuild the same missing cache entry at the same time. The resulting duplicate queries can overload databases and downstream services precisely when an application is already under pressure.

Preventing the problem requires coordinating cache population, not just storing values. Use a per-key single-flight mechanism or a suitable caching abstraction such as HybridCache. Add TTL jitter to spread expiration across keys, and consider stale-while-revalidate when slightly outdated data is acceptable.

For distributed applications, choose the coordination scope deliberately. Local locks may be sufficient for ordinary workloads, while expensive cache fills across many servers may require a distributed strategy. Measure the behavior under load and select the simplest approach that meets the application's reliability and freshness requirements.