ASP.NET Core MVC controllers and application services routinely call external payment gateways, tax/compliance endpoints, partner APIs, or internal microservices over HTTP. Instantiating new HttpClient() per request exhausts ephemeral TCP sockets and ignores DNS changes, while a single hung downstream dependency can tie up web worker threads and trigger cascading failures across your entire application.
IHttpClientFactory solves socket lifecycle and handler pooling, and Microsoft.Extensions.Resilience (built on Polly v8) layers structured fault tolerance—timeouts, retries with jitter, circuit breakers, and hedging—directly into the outbound request pipeline. This guide walks through designing a typed HTTP client, configuring standard and custom resilience pipelines with configuration binding, forwarding correlation/auth headers cleanly, and testing failure modes end-to-end in ASP.NET Core MVC.
1. Architectural Blueprint & Resilience Pipeline Ordering
In a resilience pipeline, execution order is critical. Retries wrapped outside a circuit breaker would attempt calls even after the breaker trips, whereas a per-attempt timeout placed inside a retry policy ensures each individual attempt is bounded before retrying.
Pipeline Stage | Role in Outbound HTTP | Why It Matters in Production |
Total Request Timeout | Caps the overall wall-clock time across all retry attempts. | Prevents a slow upstream dependency from holding an MVC request open indefinitely. |
Retry (Exponential + Jitter) | Re-sends transient failures (5xx, 408, 429, network errors). | Smooths brief network blips or pod restarts without thundering-herd spikes (via jitter). |
Circuit Breaker | Opens the circuit after a failure threshold is crossed; probes via half-open state. | Fails fast when a downstream service is unhealthy, protecting threads and latency. |
Attempt Timeout | Bounds a single HTTP attempt before retry evaluation. | Stops one stalled socket read from consuming the entire retry budget. |
MVC Controller / Application Service
│
▼
Typed Client (PartnerSettlementClient)
│
▼
[IHttpClientFactory] ──> Pooled HttpMessageHandler + DNS Refresh
│
▼
[Resilience Pipeline (Polly v8 via Microsoft.Extensions.Resilience)]
│ Total Timeout ──> Retry (Jitter) ──> Circuit Breaker ──> Attempt Timeout
▼
Outbound HTTP Request ──> External Service / Downstream API
2. Domain Contracts, Options, and Typed Client
Keep outbound HTTP transport details inside an infrastructure/integration client rather than leaking HttpClient or raw JSON DTOs into your MVC controllers. Here is an example for a settlement/verification gateway:
C#
using System.ComponentModel.DataAnnotations;
using System.Net;
using System.Net.Http.Json;
using System.Text.Json;
using System.Text.Json.Serialization;
public sealed class PartnerGatewayOptions
{
public const string SectionName = "Integrations:PartnerGateway";
[Required, Url]
public string BaseUrl { get; set; } = string.Empty;
[Required]
public string ApiKey { get; set; } = string.Empty;
[Range(1, 300)]
public int DefaultTimeoutSeconds { get; set; } = 15;
}
public sealed record SettlementRequest(
string ReferenceId,
string MerchantCode,
decimal Amount,
string Currency
);
public sealed record SettlementResponse(
string ReferenceId,
string Status,
string ExternalTransactionId,
DateTimeOffset ProcessedAt
);
public sealed class PartnerGatewayException : Exception
{
public HttpStatusCode? StatusCode { get; }
public string? ErrorCode { get; }
public PartnerGatewayException(
string message,
HttpStatusCode? statusCode = null,
string? errorCode = null,
Exception? innerException = null)
: base(message, innerException)
{
StatusCode = statusCode;
ErrorCode = errorCode;
}
}
public interface IPartnerGatewayClient
{
Task<SettlementResponse> SubmitSettlementAsync(
SettlementRequest request,
string idempotencyKey,
CancellationToken cancellationToken = default);
}
public sealed class PartnerGatewayClient : IPartnerGatewayClient
{
private static readonly JsonSerializerOptions SerializerOptions = new(JsonSerializerDefaults.Web)
{
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull
};
private readonly HttpClient _httpClient;
private readonly ILogger<PartnerGatewayClient> _logger;
public PartnerGatewayClient(HttpClient httpClient, ILogger<PartnerGatewayClient> logger)
{
_httpClient = httpClient;
_logger = logger;
}
public async Task<SettlementResponse> SubmitSettlementAsync(
SettlementRequest request,
string idempotencyKey,
CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(request);
ArgumentException.ThrowIfNullOrWhiteSpace(idempotencyKey);
using var httpRequest = new HttpRequestMessage(HttpMethod.Post, "v1/settlements")
{
Content = JsonContent.Create(request, options: SerializerOptions)
};
// Pass idempotency key per request so retries can safely replay the same operation
httpRequest.Headers.TryAddWithoutValidation("Idempotency-Key", idempotencyKey);
using HttpResponseMessage response = await _httpClient.SendAsync(
httpRequest,
HttpCompletionOption.ResponseHeadersRead,
cancellationToken);
if (!response.IsSuccessStatusCode)
{
string bodySnippet = await ReadSafeErrorSnippetAsync(response, cancellationToken);
_logger.LogWarning(
"Partner gateway rejected settlement {ReferenceId} with status {StatusCode}. Body: {Body}",
request.ReferenceId,
(int)response.StatusCode,
bodySnippet);
throw new PartnerGatewayException(
$"Partner gateway returned HTTP {(int)response.StatusCode} for reference '{request.ReferenceId}'.",
response.StatusCode);
}
var payload = await response.Content.ReadFromJsonAsync<SettlementResponse>(
SerializerOptions,
cancellationToken);
if (payload is null)
{
throw new PartnerGatewayException(
$"Partner gateway returned an empty response body for reference '{request.ReferenceId}'.",
response.StatusCode);
}
return payload;
}
private static async Task<string> ReadSafeErrorSnippetAsync(
HttpResponseMessage response,
CancellationToken cancellationToken)
{
try
{
string raw = await response.Content.ReadAsStringAsync(cancellationToken);
return raw.Length > 512 ? raw[..512] : raw;
}
catch
{
return "<unreadable>";
}
}
}
3. Delegating Handler for Correlation & Auth Headers
Using a DelegatingHandler keeps header injection and distributed trace propagation out of your business client and HttpClient.DefaultRequestHeaders. Note that HttpClient.DefaultRequestHeaders is shared across requests on that client instance, whereas per-request headers or correlation IDs belong in a delegating handler.
C#
using System.Diagnostics;
using Microsoft.Extensions.Options;
public sealed class PartnerAuthAndCorrelationHandler : DelegatingHandler
{
private const string CorrelationHeaderName = "X-Correlation-ID";
private readonly PartnerGatewayOptions _options;
public PartnerAuthAndCorrelationHandler(IOptions<PartnerGatewayOptions> options)
{
_options = options.Value;
}
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
if (!string.IsNullOrWhiteSpace(_options.ApiKey) && !request.Headers.Contains("X-Api-Key"))
{
request.Headers.TryAddWithoutValidation("X-Api-Key", _options.ApiKey);
}
if (!request.Headers.Contains(CorrelationHeaderName))
{
string correlationId = Activity.Current?.TraceId.ToString() ?? Guid.NewGuid().ToString("N");
request.Headers.TryAddWithoutValidation(CorrelationHeaderName, correlationId);
}
return base.SendAsync(request, cancellationToken);
}
}
4. Registering IHttpClientFactory and Microsoft.Extensions.Resilience (Program.cs)
Install Microsoft.Extensions.Http.Resilience (which brings in Microsoft.Extensions.Resilience and Polly v8 integration). You can start with AddStandardResilienceHandler() for sensible defaults, or customize retry/circuit-breaker/timeout settings directly from appsettings.json.
Sample appsettings.json
JSON
{
"Integrations": {
"PartnerGateway": {
"BaseUrl": "https://api.partner-gateway.example.com/",
"ApiKey": "replace-in-secret-store",
"DefaultTimeoutSeconds": 15
}
}
}
Startup Registration (Program.cs)
C#
using System.Net;
using Microsoft.Extensions.Http.Resilience;
using Microsoft.Extensions.Options;
using Polly;
var builder = WebApplication.CreateBuilder(args);
builder.Services
.AddOptions<PartnerGatewayOptions>()
.BindConfiguration(PartnerGatewayOptions.SectionName)
.ValidateDataAnnotations()
.ValidateOnStart();
builder.Services.AddTransient<PartnerAuthAndCorrelationHandler>();
builder.Services
.AddHttpClient<IPartnerGatewayClient, PartnerGatewayClient>((sp, client) =>
{
var opts = sp.GetRequiredService<IOptions<PartnerGatewayOptions>>().Value;
client.BaseAddress = new Uri(opts.BaseUrl, UriKind.Absolute);
client.DefaultRequestHeaders.Accept.ParseAdd("application/json");
// Let the resilience pipeline manage timeouts rather than an outer HttpClient timeout
client.Timeout = Timeout.InfiniteTimeSpan;
})
.AddHttpMessageHandler<PartnerAuthAndCorrelationHandler>()
.AddResilienceHandler("PartnerGatewayPipeline", (resilienceBuilder, context) =>
{
var opts = context.ServiceProvider.GetRequiredService<IOptions<PartnerGatewayOptions>>().Value;
var totalTimeout = TimeSpan.FromSeconds(opts.DefaultTimeoutSeconds);
// 1. Overall timeout across all attempts
resilienceBuilder.AddTimeout(totalTimeout);
// 2. Retry policy with exponential backoff + jitter for transient errors and 429 Too Many Requests
resilienceBuilder.AddRetry(new HttpRetryStrategyOptions
{
MaxRetryAttempts = 3,
Delay = TimeSpan.FromMilliseconds(250),
BackoffType = DelayBackoffType.Exponential,
UseJitter = true,
ShouldHandle = static args => ValueTask.FromResult(
HttpClientResiliencePredicates.IsTransient(args.Outcome) ||
args.Outcome.Result?.StatusCode == HttpStatusCode.TooManyRequests)
});
// 3. Circuit breaker to fail fast when downstream error rate spikes
resilienceBuilder.AddCircuitBreaker(new HttpCircuitBreakerStrategyOptions
{
SamplingDuration = TimeSpan.FromSeconds(30),
FailureRatio = 0.5,
MinimumThroughput = 10,
BreakDuration = TimeSpan.FromSeconds(15),
ShouldHandle = static args => ValueTask.FromResult(
HttpClientResiliencePredicates.IsTransient(args.Outcome) ||
args.Outcome.Result?.StatusCode == HttpStatusCode.TooManyRequests)
});
// 4. Per-attempt timeout so a stalled connection cannot exhaust the full budget
resilienceBuilder.AddTimeout(TimeSpan.FromSeconds(4));
});
builder.Services.AddControllersWithViews();
var app = builder.Build();
app.UseRouting();
app.MapControllers();
app.Run();
5. Consuming the Client in an ASP.NET Core MVC Controller
MVC controllers should stay focused on request validation, mapping to view models or JSON responses, and translating domain/gateway exceptions into clean HTTP statuses (503 Service Unavailable or 502 Bad Gateway when an upstream dependency is down or timed out).
C#
using Microsoft.AspNetCore.Mvc;
using Polly.CircuitBreaker;
using Polly.Timeout;
public sealed record CreateSettlementViewModel(
string ReferenceId,
string MerchantCode,
decimal Amount,
string Currency
);
[ApiController]
[Route("settlements")]
public sealed class SettlementsController : ControllerBase
{
private readonly IPartnerGatewayClient _gatewayClient;
private readonly ILogger<SettlementsController> _logger;
public SettlementsController(
IPartnerGatewayClient gatewayClient,
ILogger<SettlementsController> logger)
{
_gatewayClient = gatewayClient;
_logger = logger;
}
[HttpPost]
public async Task<IActionResult> Create(
[FromBody] CreateSettlementViewModel model,
CancellationToken cancellationToken)
{
if (!ModelState.IsValid)
{
return ValidationProblem(ModelState);
}
// Derive a deterministic idempotency key from the business reference so retries are safe
string idempotencyKey = $"settlement:{model.MerchantCode}:{model.ReferenceId}";
var request = new SettlementRequest(
model.ReferenceId,
model.MerchantCode,
model.Amount,
model.Currency);
try
{
SettlementResponse result = await _gatewayClient.SubmitSettlementAsync(
request,
idempotencyKey,
cancellationToken);
return Ok(result);
}
catch (BrokenCircuitException ex)
{
_logger.LogWarning(ex, "Circuit open for partner gateway on settlement {ReferenceId}.", model.ReferenceId);
return StatusCode(StatusCodes.Status503ServiceUnavailable, new
{
Error = "Upstream settlement service is temporarily unavailable. Please retry shortly."
});
}
catch (TimeoutRejectedException ex)
{
_logger.LogWarning(ex, "Partner gateway timed out on settlement {ReferenceId}.", model.ReferenceId);
return StatusCode(StatusCodes.Status504GatewayTimeout, new
{
Error = "Upstream settlement service timed out."
});
}
catch (PartnerGatewayException ex) when (ex.StatusCode is HttpStatusCode.BadRequest or HttpStatusCode.UnprocessableEntity)
{
return UnprocessableEntity(new
{
Error = "Settlement request rejected by partner gateway.",
Details = ex.Message
});
}
catch (PartnerGatewayException ex)
{
_logger.LogError(ex, "Partner gateway failure for settlement {ReferenceId}.", model.ReferenceId);
return StatusCode(StatusCodes.Status502BadGateway, new
{
Error = "Failed to communicate with upstream settlement service."
});
}
}
}
6. Testing Retries and Upstream Failures
Because IHttpClientFactory builds outbound pipelines on top of HttpMessageHandler, you can test retry behavior and controller error mapping deterministically using a custom DelegatingHandler (or HttpMessageHandler stub) in WebApplicationFactory<Program>.
C#
using System.Net;
using System.Net.Http.Json;
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.Extensions.DependencyInjection;
using Xunit;
public sealed class SettlementsControllerTests
{
private sealed class FlakyUpstreamHandler : HttpMessageHandler
{
private int _attemptCount;
public int AttemptCount => _attemptCount;
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
int current = Interlocked.Increment(ref _attemptCount);
if (current <= 2)
{
return Task.FromResult(new HttpResponseMessage(HttpStatusCode.ServiceUnavailable));
}
var payload = new SettlementResponse(
ReferenceId: "REF-9001",
Status: "Approved",
ExternalTransactionId: "EXT-12345",
ProcessedAt: DateTimeOffset.UtcNow);
return Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK)
{
Content = JsonContent.Create(payload)
});
}
}
[Fact]
public async Task Create_Transient503TwiceThen200_RetriesAndSucceeds()
{
var flakyHandler = new FlakyUpstreamHandler();
await using var factory = new WebApplicationFactory<Program>().WithWebHostBuilder(builder =>
{
builder.ConfigureServices(services =>
{
// Replace primary handler for IPartnerGatewayClient in tests
services.ConfigureHttpClientDefaults(clientBuilder =>
{
clientBuilder.ConfigurePrimaryHttpMessageHandler(() => flakyHandler);
});
});
});
using var client = factory.CreateClient();
var model = new CreateSettlementViewModel("REF-9001", "MERCH-01", 250.00m, "USD");
HttpResponseMessage response = await client.PostAsJsonAsync("/settlements", model);
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
Assert.True(flakyHandler.AttemptCount >= 3);
}
}
7. Conclusion
Combining IHttpClientFactory with Microsoft.Extensions.Resilience gives ASP.NET Core MVC applications disciplined control over outbound HTTP connections and downstream failures. Pooled handlers prevent socket exhaustion, ordered resilience strategies (timeouts, jittered retries, and circuit breakers) stop transient blips from cascading into outages, and clean exception mapping in controllers keeps client responses predictable when external services degrade.
Join the conversation! Your thoughts help the community grow.