100 Concurrent Requests. One Expired Cache Entry. Why Does Your Database Execute 100 Queries?

Shubham Sidnale

Understanding Cache Stampede and How HybridCache Solves It in ASP.NET Core

Introduction

Imagine you're developing an e-commerce application using ASP.NET Core Web API, Entity Framework Core, SQL Server, and Redis.

Your product details endpoint is performing well.

You've implemented caching to reduce unnecessary database queries.

Everything looks good until your application receives a sudden spike in traffic.

At that exact moment, a popular product's cache entry expires.

Now 100 users request the same product simultaneously.

What happens next?

All 100 requests check the cache.

All 100 requests discover that the cached entry is missing.

And without proper coordination, all 100 requests execute the same SQL query.

Your database suddenly processes 100 identical queries to retrieve information that only needs to be fetched once.

This problem is known as a Cache Stampede, sometimes called the Thundering Herd Problem.

Let's understand why it happens, how it affects application performance, and how Microsoft's HybridCache can help solve it.

1. What Is a Cache Stampede?

A cache stampede occurs when multiple concurrent requests attempt to regenerate the same missing or expired cached data.

Consider an API endpoint:

GET /api/products/101

The product details are stored in Redis with a 10-minute expiration.

While the entry exists, requests are served from Redis.

But when the cache entry expires, multiple requests may reach the database simultaneously.

Without Cache Stampede Protection

          100 Concurrent Requests
                    |
                    v
               Redis Cache
                    |
               CACHE MISS
                    |
        +-----------+-----------+
        |           |           |
     Request 1   Request 2   Request 100
        |           |           |
        v           v           v
     SQL Query   SQL Query   SQL Query
        |           |           |
        +-----------+-----------+
                    |
                    v
                SQL Server

        100 requests = up to 100 queries

Every request independently tries to repopulate the cache.

This creates unnecessary database work.

Why Is This Dangerous?

A cache stampede can cause:

Sudden spikes in database connections
Increased SQL Server CPU utilization
Higher API response times
Database connection pool pressure
Request timeouts
Reduced application throughput

In extreme cases, a database slowdown can trigger additional request retries, increasing pressure on an already overloaded system.

The important point is that caching alone doesn't automatically prevent a cache stampede.

2. Traditional Cache-Aside Implementation

Let's look at a common implementation using IDistributedCache.

public async Task<Product?> GetProductAsync(
    int productId,
    CancellationToken cancellationToken)
{
    string cacheKey = $"product:{productId}";

    var cachedProduct = await _cache.GetStringAsync(
        cacheKey,
        cancellationToken);

    if (cachedProduct != null)
    {
        return JsonSerializer.Deserialize<Product>(
            cachedProduct);
    }

    var product = await _dbContext.Products
        .AsNoTracking()
        .FirstOrDefaultAsync(
            p => p.Id == productId,
            cancellationToken);

    if (product != null)
    {
        await _cache.SetStringAsync(
            cacheKey,
            JsonSerializer.Serialize(product),
            new DistributedCacheEntryOptions
            {
                AbsoluteExpirationRelativeToNow =
                    TimeSpan.FromMinutes(10)
            },
            cancellationToken);
    }

    return product;
}

This implementation works correctly for ordinary cache hits and misses.

However, it doesn't coordinate concurrent cache misses.

If 100 requests check Redis before any request has populated the entry, they can all proceed to SQL Server.

The cache is eventually populated, but the database has already performed duplicate work.

This is the limitation we want to address.

3. The Solution: HybridCache in ASP.NET Core

Microsoft introduced HybridCache to simplify caching scenarios that need more than basic cache reads and writes.

HybridCache provides:

A primary in-memory cache
An optional secondary distributed cache
Stampede protection for concurrent requests
Cache entry expiration
Tag-based invalidation
Asynchronous APIs

The most interesting feature for our scenario is Stampede Protection.

When multiple callers request the same missing cache key within one application instance, HybridCache can coordinate them so that the underlying data factory executes once.

Other callers await the shared operation instead of independently querying the database.

Simple Analogy

Imagine 100 people standing in front of a lift.

All 100 people want to go to the same floor.

Without coordination, imagine everyone requesting a separate lift.

With coordination, one lift arrives and the group shares the ride.

HybridCache applies a similar idea to concurrent cache misses.

4. Implement HybridCache in ASP.NET Core

Let's implement HybridCache using ASP.NET Core Web API and Entity Framework Core.

Step 1: Install HybridCache

dotnet add package Microsoft.Extensions.Caching.Hybrid

Step 2: Register HybridCache

In Program.cs:

builder.Services.AddHybridCache();

This registers the HybridCache services.

For a basic implementation, you can start with the in-memory caching layer.

Step 3: Implement the Product Service

using Microsoft.Extensions.Caching.Hybrid;
using Microsoft.EntityFrameworkCore;

public class ProductService
{
    private readonly HybridCache _cache;
    private readonly AppDbContext _dbContext;

    public ProductService(
        HybridCache cache,
        AppDbContext dbContext)
    {
        _cache = cache;
        _dbContext = dbContext;
    }

    public async Task<ProductDto?> GetProductAsync(
        int productId,
        CancellationToken cancellationToken)
    {
        string cacheKey = $"product:{productId}";

        return await _cache.GetOrCreateAsync(
            cacheKey,
            async ct =>
            {
                return await _dbContext.Products
                    .AsNoTracking()
                    .Where(p => p.Id == productId)
                    .Select(p => new ProductDto
                    {
                        Id = p.Id,
                        Name = p.Name,
                        Price = p.Price
                    })
                    .FirstOrDefaultAsync(ct);
            },
            cancellationToken: cancellationToken);
    }
}

What Happens Internally?

Suppose 100 requests arrive for product 101 after its cache entry expires.

First request:

Checks HybridCache.
Cache entry is missing.
Executes the EF Core query.
Stores the result in the cache.
Returns the product.

Other concurrent requests:

Check the same cache key.
Find that the cache population operation is already running.
Await the shared operation.
Receive its result without independently executing the same query.

For overlapping misses on the same key within the same application instance, the factory is coordinated.

100 concurrent requests can therefore result in one database query instead of 100.

This assumes the requests share the same HybridCache instance and the operation is eligible for the normal coordinated cache path.

5. How HybridCache Works with Redis

HybridCache supports two caching levels.

L1 Cache — In-Memory Cache

Stores frequently accessed data inside the application's process.

This avoids network calls for local cache hits.

L2 Cache — Distributed Cache

Uses a distributed caching provider such as Redis.

This allows multiple application instances to access shared cached data.

Install Redis Integration

dotnet add package Microsoft.Extensions.Caching.StackExchangeRedis

Configure Program.cs

builder.Services.AddStackExchangeRedisCache(options =>
{
    options.Configuration =
        builder.Configuration.GetConnectionString("Redis");
});

builder.Services.AddHybridCache(options =>
{
    options.DefaultEntryOptions =
        new HybridCacheEntryOptions
        {
            Expiration = TimeSpan.FromMinutes(10),
            LocalCacheExpiration = TimeSpan.FromMinutes(2)
        };
});

appsettings.json

{
  "ConnectionStrings": {
    "Redis": "localhost:6379"
  }
}

Request Flow

             ASP.NET Core API
                    |
                    v
              HybridCache
                    |
                    v
          L1: In-Memory Cache
                    |
                Cache Miss
                    |
                    v
             L2: Redis Cache
                    |
                Cache Miss
                    |
                    v
                SQL Server
                    |
                    v
             Populate Cache
                    |
                    v
              Return Result

If the data exists in L1, HybridCache can return it without contacting Redis.

If L1 misses but L2 contains the value, the value can be loaded from Redis and reused locally.

If both levels miss, the factory executes to retrieve the data.

6. Important Limitation: Stampede Protection Is Per Application Instance

Now let's consider a distributed application.

Suppose your API is deployed across five servers behind a load balancer.

Each server has its own HybridCache instance.

If the same product expires across all servers, concurrent requests may trigger one factory execution per server.

For example:

              Load Balancer
                    |
       +------------+------------+
       |            |            |
    Server A     Server B     Server C
       |            |            |
    1 Query      1 Query      1 Query
       |            |            |
       +------------+------------+
                    |
                SQL Server

HybridCache coordinates concurrent factory execution within an application instance.

It doesn't provide a distributed lock across all application instances.

Therefore, with five servers, multiple queries may still execute for the same missing key.

A shared Redis cache can reduce the likelihood of duplicate database queries, but it doesn't guarantee global single-flight behavior.

For applications that require coordination across servers, additional distributed coordination or locking mechanisms may be necessary.

7. What Happens When Cached Data Changes?

Suppose product 101 has the following price:

Product: Laptop
Price: ₹50,000

Later, an administrator updates the price to ₹48,000.

The database now contains the updated value, but the cache may still contain the previous price.

This is a cache consistency problem.

Solution: Invalidate the Cache

After successfully updating the product, invalidate its cache entry.

await _cache.RemoveAsync(
    $"product:{productId}",
    cancellationToken);

The next request can repopulate the cache with the updated data.

Tag-Based Invalidation

HybridCache also supports tagging entries.

return await _cache.GetOrCreateAsync(
    $"product:{productId}",
    async ct =>
    {
        return await LoadProductAsync(productId, ct);
    },
    tags: ["products"],
    cancellationToken: cancellationToken);

You can then invalidate a group of entries:

await _cache.RemoveByTagAsync(
    "products",
    cancellationToken);

This is useful when many cached entries belong to the same logical group.

Important: Tag invalidation is logical invalidation. It doesn't necessarily physically delete every tagged entry immediately.

Also, distributed invalidation doesn't instantly remove every in-memory copy held by other application instances.

For strict cross-server consistency, additional design considerations are necessary.

8. What Happens If a User Cancels the Request?

HybridCache supports cancellation tokens.

Consider 100 callers waiting for the same cache population operation.

If one user disconnects or cancels their request, it shouldn't necessarily cancel the shared operation for the other 99 callers.

HybridCache coordinates cancellation for the shared factory operation.

The cancellation token supplied to the factory represents the combined cancellation state of participating callers.

This helps avoid one caller unnecessarily terminating work that other callers still need.

However, the factory and downstream dependencies must properly support cancellation for it to be effective.

9. HybridCache vs. IMemoryCache vs. IDistributedCache

Feature	IMemoryCache	IDistributedCache	HybridCache
Local in-memory cache	Yes	No	Yes
Distributed cache support	No	Yes	Optional
Built-in stampede protection	No	No	Yes
Built-in tag invalidation	No	No	Yes
Get-or-create API	Yes	No	Yes
Suitable for multiple servers	Local only	Yes	Yes, with distributed provider

HybridCache is particularly useful when an application benefits from both local and distributed caching and needs coordinated cache population.

However, it isn't automatically the best choice for every application.

For low-traffic applications with inexpensive queries, IMemoryCache may be sufficient.

10. How to Verify That Stampede Protection Works

Don't rely only on theory. Validate the behavior through load testing.

For example, use k6, NBomber, or another load-testing tool.

Test Scenario

Create a product endpoint using IDistributedCache.
Configure a short cache expiration.
Ensure the cache key is absent.
Send 100 overlapping requests for the same product.
Count how many times the database query executes.
Repeat the experiment using HybridCache.

Expected Behavior

Scenario	Database factory executions
Basic cache-aside with 100 overlapping misses	Potentially up to 100
HybridCache, one instance, 100 overlapping misses	Typically 1
HybridCache, five instances, shared miss	Potentially multiple, up to one per instance for a single overlapping miss wave

These are conceptual expectations, not measured benchmark results.

Actual behavior depends on request timing, cache state, concurrency, failures, and deployment architecture.

For reliable results, instrument the data-loading factory and repeat the test under controlled conditions.

11. Senior .NET Developer Interview Questions

Q1. What is a cache stampede?

A cache stampede occurs when many concurrent requests encounter the same missing or expired cache entry and independently execute expensive operations to regenerate it.

Q2. Does Redis automatically prevent a cache stampede?

No. Redis stores cached data, but a basic cache-aside implementation doesn't automatically coordinate concurrent cache misses.

Q3. How does HybridCache prevent cache stampede?

HybridCache coordinates overlapping cache population operations for the same key within an application instance, allowing callers to share the result.

Q4. Does HybridCache guarantee only one database query across multiple servers?

No. Its built-in stampede protection is local to each application instance.

Q5. What is the difference between L1 and L2 caching?

L1 is the application's local in-memory cache.

L2 is an optional shared distributed cache, such as Redis.

Q6. How do you handle stale data in HybridCache?

Use appropriate expiration policies, explicit key invalidation, tag-based invalidation, and additional consistency mechanisms when required.

Q7. When would you avoid HybridCache?

For simple, low-traffic applications where local caching meets the requirements, introducing a distributed cache or additional caching complexity may not provide meaningful benefits.

Conclusion

Caching improves application performance by reducing repeated database queries.

But caching introduces a new challenge: What happens when many requests encounter the same missing entry simultaneously?

Without coordination, a single expired cache key can trigger a sudden wave of identical database queries.

HybridCache addresses this problem through built-in stampede protection, while also providing local caching, optional distributed caching, expiration, and invalidation capabilities.

However, it's important to understand its boundaries.

HybridCache coordinates concurrent cache population within an application instance. It doesn't automatically guarantee one database query across an entire distributed deployment.

Key Takeaway

Caching reduces repeated queries. Stampede protection prevents concurrent cache misses from creating unnecessary duplicate work.

Before scaling your database to handle traffic spikes, investigate whether your application is repeatedly executing the same expensive operations.

Sometimes, the biggest performance improvement comes not from making SQL faster, but from avoiding unnecessary SQL queries altogether.

Official References

Microsoft Learn — HybridCache in ASP.NET Core
https://learn.microsoft.com/en-us/aspnet/core/performance/caching/hybrid
Microsoft Learn — Distributed Caching in ASP.NET Core
https://learn.microsoft.com/en-us/aspnet/core/performance/caching/distributed
Microsoft Learn — In-Memory Caching in ASP.NET Core
https://learn.microsoft.com/en-us/aspnet/core/performance/caching/memory
Microsoft Learn — HybridCache API Reference
https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.caching.hybrid.hybridcache

#DotNet #ASPNETCore #HybridCache #Redis #Caching #CacheStampede #CSharp #BackendDevelopment #PerformanceOptimization #Microservices #SoftwareArchitecture

ASP.NET CoreHybridCachecache stampedeRedis cachingEntity Framework Coredistributed caching

Comments

Join the conversation! Your thoughts help the community grow.