In distributed systems, the same request is often sent more than once. A mobile application may retry after a timeout, an API gateway may automatically retry failed requests, or a message broker may redeliver a message after a consumer crash. Without proper safeguards, these duplicate requests can create duplicate orders, process multiple payments, send repeated emails, or corrupt business data.
This is where idempotency becomes essential. An idempotent API guarantees that processing the same request multiple times produces the same result as processing it once. Rather than relying on clients to avoid retries, the server safely handles duplicate requests while maintaining data consistency.
This article demonstrates how to build production-ready idempotent ASP.NET Core APIs using the Idempotency-Key pattern, distributed caching, and transactional persistence.
Why Duplicate Requests Happen
Duplicate requests are normal in distributed systems and should be expected.
Common scenarios include:
HTTP client retries after timeout
Browser refresh during form submission
API Gateway retry policies
Reverse proxy retries
Mobile network instability
Message broker redelivery
User clicking the Submit button multiple times
Consider an order creation endpoint:
POST /api/orders
If the client retries because the response was lost, the server may create two orders even though the customer intended to create only one.
The problem is rarely visible during development but becomes common in production.
Idempotency vs Retry
These concepts are related but different.
| Retry | Idempotency |
|---|
| Client sends request again | Server safely handles duplicate requests |
| Improves reliability | Prevents duplicate processing |
| Client responsibility | Server responsibility |
| May execute operation multiple times | Guarantees only one successful execution |
Retries without idempotency increase the likelihood of duplicate data.
Typical Request Flow
sequenceDiagram
participant Client
participant API
participant Cache
participant Database
Client->>API: POST /orders + Idempotency-Key
API->>Cache: Check Key
alt Key Exists
Cache-->>API: Previous Response
API-->>Client: Return Cached Result
else New Request
API->>Database: Save Order
API->>Cache: Store Response
API-->>Client: Success
end
The API never processes the same key twice.
Choosing an Idempotency Key
Clients generate a unique identifier for every logical request.
Example:
Idempotency-Key:
9f4a8ef5-5d0d-4fd8-a94f-c58ab7ef41b3
Good choices include:
GUID
UUID
Secure random identifier
Avoid:
Timestamp
Username
Order Number
Email Address
The key should identify a single business operation—not a user.
Designing the API Contract
POST /api/orders
Headers
Idempotency-Key:
9f4a8ef5-5d0d-4fd8-a94f-c58ab7ef41b3
If the same request arrives again with the same key:
Clients receive identical results regardless of how many retries occur.
Building an Idempotency Middleware
Instead of duplicating logic inside every controller, centralize it.
public class IdempotencyMiddleware
{
private readonly RequestDelegate _next;
private readonly IDistributedCache _cache;
public IdempotencyMiddleware(
RequestDelegate next,
IDistributedCache cache)
{
_next = next;
_cache = cache;
}
public async Task Invoke(HttpContext context)
{
if (!context.Request.Headers.TryGetValue(
"Idempotency-Key",
out var key))
{
await _next(context);
return;
}
var cachedResponse =
await _cache.GetStringAsync(key!);
if (cachedResponse != null)
{
context.Response.ContentType =
"application/json";
await context.Response.WriteAsync(
cachedResponse);
return;
}
var originalBody = context.Response.Body;
using var memoryStream =
new MemoryStream();
context.Response.Body = memoryStream;
await _next(context);
memoryStream.Position = 0;
var response =
await new StreamReader(memoryStream)
.ReadToEndAsync();
await _cache.SetStringAsync(
key!,
response,
new DistributedCacheEntryOptions
{
AbsoluteExpirationRelativeToNow =
TimeSpan.FromHours(24)
});
memoryStream.Position = 0;
await memoryStream.CopyToAsync(originalBody);
context.Response.Body = originalBody;
}
}
This middleware intercepts duplicate requests before controller logic executes.
Registering the Middleware
builder.Services.AddStackExchangeRedisCache(options =>
{
options.Configuration =
builder.Configuration
.GetConnectionString("Redis");
});
app.UseMiddleware<IdempotencyMiddleware>();
Distributed caching allows every API instance to share the same idempotency store.
Why In-Memory Cache Is Not Enough
Many developers begin with:
IMemoryCache
This works only on a single server.
Consider two application instances behind a load balancer.
Client
│
Load Balancer
┌──────┴──────┐
API 1 API 2
If the retry reaches a different server, the duplicate request is processed again.
Production systems should use:
Redis
SQL Server
Cosmos DB
PostgreSQL
DynamoDB
The idempotency store must be shared across all instances.
Handling Database Consistency
Caching alone does not guarantee correctness.
Imagine:
Order inserted into SQL Server.
Redis fails.
Retry arrives.
Second order inserted.
The cache missed the first request because persistence failed.
The database should also enforce uniqueness.
Example:
CREATE UNIQUE INDEX
IX_IdempotencyKey
ON Orders(IdempotencyKey);
Even if the cache becomes unavailable, the database prevents duplicates.
Multiple layers of protection produce resilient systems.
Expiration Strategy
Should idempotency keys live forever?
Usually not.
Typical expiration:
| Business Scenario | TTL |
|---|
| Payment APIs | 24 Hours |
| Order Processing | 24–48 Hours |
| Inventory Updates | Few Hours |
| Financial Transactions | Business Requirement |
Keeping keys indefinitely increases storage costs without providing additional value.
Common Production Failures
| Problem | Root Cause | Solution |
|---|
| Duplicate Orders | Missing idempotency key | Reject request |
| Cache Misses | Short expiration | Increase TTL |
| Duplicate Processing | Local memory cache | Use distributed cache |
| Race Conditions | Concurrent requests | Use distributed locking or database constraints |
| Different Request Body Same Key | Client bug | Validate request payload before returning cached response |
Security Considerations
Do not trust arbitrary client keys.
Recommended practices:
Validate key length.
Limit maximum size.
Reject malformed identifiers.
Apply rate limiting.
Require authentication.
Encrypt sensitive cached responses if necessary.
Never expose internal cache identifiers.
Attackers can intentionally reuse keys to probe API behavior.
Best Practices
Generate a new idempotency key for every business operation.
Store both the request fingerprint and the response.
Use Redis or another distributed cache.
Back idempotency with a unique database constraint.
Log duplicate requests for monitoring.
Expire keys based on business requirements.
Return the original HTTP status code and response body for duplicate requests.
Common Anti-Patterns
Avoid these implementation mistakes:
Using timestamps as idempotency keys.
Relying only on client-side retry prevention.
Using IMemoryCache in load-balanced environments.
Returning different responses for duplicate requests.
Creating a new database record before checking the idempotency key.
Ignoring concurrent requests with the same key.
These mistakes often remain hidden until the application experiences production traffic.
When Should You Use Idempotency?
Idempotency is highly recommended for:
Read-only GET endpoints generally do not require additional idempotency logic because they are naturally idempotent.
FAQ
Does every POST endpoint need idempotency?
No. Apply it to operations where duplicate execution could create incorrect business results or financial loss.
Can PUT requests still benefit from idempotency?
Yes. Although PUT is defined as idempotent by HTTP semantics, complex business workflows may still require server-side safeguards.
Should I store only the key?
No. Store the key, request fingerprint (or hash), response payload, status code, and expiration metadata. This ensures that a reused key with a different payload can be detected and rejected.
Is Redis mandatory?
No. Any shared, durable store can work, but Redis is commonly chosen because of its low latency and built-in expiration support.
Conclusion
Idempotency is a fundamental requirement for reliable distributed systems—not an optional enhancement. Network failures, client retries, and message redelivery are inevitable, and APIs must be designed to handle them safely.
A robust implementation combines Idempotency-Key validation, distributed caching, and database-level uniqueness constraints. This layered approach prevents duplicate processing, supports horizontal scaling, and maintains data consistency even when components fail.
By treating duplicate requests as a normal part of distributed computing rather than an exceptional case, you can build ASP.NET Core APIs that remain reliable, predictable, and resilient under real-world production conditions.