Networking  

Building DNS-Aware Failover Clients with .NET 11

Modern distributed applications rarely communicate with a single fixed server.

A production API may sit behind a load balancer, service discovery layer, DNS-based failover system, or cloud infrastructure that changes endpoint addresses over time.

That creates a subtle networking problem for long-running .NET applications.

A service can continue using an established HTTP connection even after DNS has changed.

Consider this simplified architecture:

Application
    ↓
api.example.internal
    ↓
10.10.10.20

Later, the infrastructure changes:

api.example.internal
    ↓
10.10.10.35

New DNS resolution may return the new address, but an existing connection can remain associated with the previous endpoint.

For short-lived processes this may not matter.

For long-running APIs, workers, and background services, stale connections can become a real availability problem.

The solution is not to resolve DNS manually before every request. A better approach is to build an HTTP client strategy that understands DNS changes, connection lifetime, failure detection, and controlled endpoint rotation.

Why DNS Changes Are Difficult for Long-Running Clients

DNS is commonly treated as:

Name
 ↓
IP address

But an HTTP client has more state than that.

A simplified connection lifecycle looks like:

DNS lookup
    ↓
TCP connection
    ↓
TLS handshake
    ↓
HTTP requests
    ↓
Connection reuse

If the DNS record changes after the connection is established, the client may continue using the existing connection.

That means:

DNS
10.10.10.20 → 10.10.10.35

does not automatically imply:

Existing TCP connection
10.10.10.20 → 10.10.10.35

The connection has its own lifecycle.

Why HttpClient Lifetime Matters

A common .NET recommendation is to avoid creating a new HttpClient for every request.

Instead, applications typically reuse clients through IHttpClientFactory or another managed lifetime strategy.

For example:

builder.Services.AddHttpClient<OrdersClient>();

This provides connection reuse and centralized configuration.

But connection reuse creates another question:

How long should an established connection remain valid?

If connections remain alive indefinitely, DNS changes may take longer to influence actual network traffic.

Configure Connection Lifetime

SocketsHttpHandler provides connection lifetime controls.

For example:

var handler = new SocketsHttpHandler
{
    PooledConnectionLifetime = TimeSpan.FromMinutes(5)
};

var client = new HttpClient(handler);

The idea is straightforward:

Connection created
      ↓
Used for requests
      ↓
Lifetime expires
      ↓
Connection removed from pool
      ↓
New connection created
      ↓
DNS resolution can occur again

This provides a controlled way to allow DNS changes to eventually influence new connections.

Why Not Use a Very Short Lifetime?

You might be tempted to configure:

PooledConnectionLifetime = TimeSpan.FromSeconds(1);

That can undermine connection pooling.

Every new connection can require:

DNS
 ↓
TCP
 ↓
TLS

For HTTPS services, repeated TLS handshakes can add CPU consumption and latency.

The objective is therefore not:

Refresh DNS as fast as possible

but:

Balance connection reuse with endpoint freshness

DNS-Aware Failover Architecture

A resilient client can be designed around several layers:

                  HTTP Client
                      |
              Connection Pool
                      |
          +-----------+-----------+
          |                       |
     Connection Age          Request Failure
          |                       |
          v                       v
   Rotate connection       Retry policy
          |                       |
          +-----------+-----------+
                      |
                 New connection
                      |
                 DNS resolution

The connection pool handles normal endpoint rotation.

The retry layer handles transient failures.

These are different responsibilities.

Do Not Treat Retries as DNS Refresh

Suppose a request fails because an endpoint has disappeared.

A naive retry might simply send the request again over another existing connection.

That does not necessarily solve the underlying endpoint problem.

A better strategy is:

Request
  ↓
Failure
  ↓
Determine failure type
  ↓
Allow connection rotation
  ↓
Retry when safe

Retries should not become an excuse to ignore connection lifecycle management.

Configure HttpClientFactory

For ASP.NET Core applications, a named or typed client is usually easier to manage.

For example:

builder.Services.AddHttpClient<PaymentsClient>()
    .ConfigurePrimaryHttpMessageHandler(() =>
        new SocketsHttpHandler
        {
            PooledConnectionLifetime = TimeSpan.FromMinutes(5),
            PooledConnectionIdleTimeout = TimeSpan.FromMinutes(2)
        });

The exact values should be based on the DNS TTL, infrastructure behavior, traffic pattern, and acceptable connection churn.

There is no universal five-minute setting that is correct for every system.

DNS TTL and Connection Lifetime Are Different

DNS TTL tells resolvers how long a DNS answer can be cached.

Connection lifetime controls how long an established HTTP connection can remain in the pool.

They are related, but they are not the same thing.

For example:

DNS TTL:                60 seconds
Connection lifetime:   10 minutes

The application may continue using an existing connection beyond the DNS record's TTL.

Conversely:

DNS TTL:                10 minutes
Connection lifetime:   30 seconds

the application may establish connections more frequently than necessary.

The configuration should therefore consider both infrastructure behavior and application traffic.

Use Failure Signals as Part of Failover

Connection lifetime alone does not make a client highly available.

Consider:

Primary endpoint
      ↓
Connection established
      ↓
Endpoint becomes unhealthy

Waiting for a five-minute connection lifetime may be too slow.

Applications should therefore detect meaningful failures and respond appropriately.

Useful signals include:

  • Connection reset

  • Connection refused

  • Timeout

  • DNS resolution failure

  • HTTP 5xx responses

  • Service-specific health failures

However, not every HTTP error should trigger a retry.

A 400 Bad Request is normally an application error, not a transient infrastructure failure.

Retry Only Safe Operations

Retries can create duplicate operations.

For example:

POST /payments

might successfully reach the server even though the client does not receive the response.

If the client blindly retries:

POST /payments
POST /payments

the operation could be performed twice.

For idempotent operations such as many GET requests, retrying transient failures is generally easier to reason about.

For non-idempotent operations, use application-level idempotency mechanisms where appropriate.

Combine DNS Rotation With Retry Policies

A resilient architecture can look like:

             HTTP Request
                   |
                   v
             Existing Pool
                   |
              Request fails
                   |
          +--------+--------+
          |                 |
      Retryable?         Not retryable
          |                 |
         Yes                |
          |                 |
    Retry policy            Return error
          |
          v
    New connection
          |
          v
    Fresh DNS resolution

This is much safer than retrying every failed request.

Handle DNS Resolution Failures

DNS itself can fail.

For example:

Application
    ↓
DNS resolver
    ↓
Temporary failure

A client should not immediately assume that the service itself is unavailable.

Monitoring should distinguish between:

DNS failure
Connection failure
TLS failure
HTTP failure
Application failure

These categories can require different responses.

Add Structured Logging

Networking failures are difficult to troubleshoot without context.

A request log should ideally include:

Request ID
Host
HTTP method
Status code
Elapsed time
Exception type
Retry count
Connection-related error

For example:

logger.LogWarning(
    exception,
    "HTTP request failed for {Host} after {ElapsedMs} ms",
    host,
    elapsedMilliseconds);

Avoid logging sensitive request or authentication data.

Measure Connection Behavior

If DNS failover is important, observability should extend beyond HTTP status codes.

Track:

  • Connection creation

  • Connection reuse

  • Connection failures

  • DNS resolution errors

  • Request latency

  • Retry count

  • Timeout count

  • HTTP status distribution

A useful metric is:

Requests using old endpoint
Requests using new endpoint

during a controlled failover test.

That tells you whether your connection-lifetime strategy actually works.

Build a Failover Test Environment

Do not wait for a production outage to test DNS behavior.

Create an environment where the hostname can switch between endpoints.

For example:

api.internal
    |
    +---- Endpoint A
    |
    +---- Endpoint B

Start with Endpoint A.

Then switch the DNS mapping:

Before:
api.internal → A

After:
api.internal → B

Continue generating requests and observe:

How long does traffic remain on A?
When does traffic begin reaching B?
Are requests failing during the transition?
How many retries occur?

This produces meaningful operational data.

Test With Long-Lived Traffic

A failover test should run long enough to expose connection reuse.

For example:

Start application
      ↓
Create HTTP connections
      ↓
Generate continuous traffic
      ↓
Change DNS
      ↓
Continue traffic
      ↓
Measure endpoint transition

A short test that creates a new client for every request will hide the problem.

Benchmark Different Connection Lifetimes

A practical experiment can compare:

30 seconds
1 minute
5 minutes
10 minutes
30 minutes

Measure:

  • Failover detection time

  • Connection creation rate

  • TLS handshake rate

  • CPU

  • Request latency

  • Error rate

You may discover that a five-minute setting is appropriate for one workload while a 30-second setting is excessive for another.

Consider Connection Idle Timeout

Connection lifetime and idle timeout solve different problems.

PooledConnectionLifetime controls how long a connection can remain in the pool.

PooledConnectionIdleTimeout controls how long an unused connection can remain there.

For example:

var handler = new SocketsHttpHandler
{
    PooledConnectionLifetime = TimeSpan.FromMinutes(5),
    PooledConnectionIdleTimeout = TimeSpan.FromMinutes(2)
};

This allows active connections to follow the configured lifetime while unused connections are removed sooner.

Avoid Creating HttpClient Per Request

This pattern is problematic:

public async Task<string> GetAsync()
{
    using var client = new HttpClient();

    return await client.GetStringAsync(url);
}

It can lead to unnecessary connection creation and resource pressure.

Prefer managed client reuse:

public sealed class OrdersClient
{
    private readonly HttpClient _client;

    public OrdersClient(HttpClient client)
    {
        _client = client;
    }

    public Task<string> GetOrdersAsync()
    {
        return _client.GetStringAsync("/orders");
    }
}

Register it through dependency injection:

builder.Services.AddHttpClient<OrdersClient>();

Then configure connection behavior centrally.

Do Not Depend on DNS Alone for Service Discovery

DNS is useful for endpoint discovery, but it should not be the only availability mechanism in every architecture.

Large distributed systems may use:

  • Load balancers

  • Service registries

  • Container orchestration

  • Mesh-based routing

  • Cloud-native service discovery

The client strategy should match the infrastructure.

If the platform already provides service discovery and health-aware routing, duplicating that logic in every application can create unnecessary complexity.

Common Mistakes

Setting an Extremely Short Connection Lifetime

This can create unnecessary TCP and TLS overhead.

Setting an Extremely Long Lifetime

Endpoint changes may take too long to affect active traffic.

Retrying Every HTTP Error

Application errors are not automatically transient network failures.

Creating HttpClient Per Request

This defeats connection pooling and can create resource pressure.

Ignoring DNS TTL

Connection settings should be designed with actual infrastructure behavior in mind.

Testing Only New Connections

That does not validate the behavior of long-running clients.

Failing to Monitor Endpoint Changes

Without observability, it is difficult to prove that failover actually worked.

Best Practices

  1. Reuse HttpClient instances through managed dependency injection.

  2. Configure PooledConnectionLifetime based on workload and DNS behavior.

  3. Use PooledConnectionIdleTimeout to remove unused connections.

  4. Separate connection rotation from request retry logic.

  5. Retry only transient and safe operations.

  6. Use idempotency mechanisms for operations that may be retried.

  7. Distinguish DNS, connection, TLS, HTTP, and application failures.

  8. Add structured networking telemetry.

  9. Test DNS failover with long-lived clients.

  10. Measure endpoint transition time instead of assuming it works.

  11. Benchmark connection-lifetime settings under realistic traffic.

  12. Use the infrastructure's native service-discovery capabilities where appropriate.

Frequently Asked Questions

Does changing DNS immediately move existing HTTP traffic?

No. An existing TCP connection is independent of a new DNS lookup. The application may continue using an established connection until it is closed or removed from the pool.

Does PooledConnectionLifetime force a DNS lookup?

It causes the connection to be discarded after its configured lifetime, allowing a subsequent connection establishment to perform normal endpoint resolution. The exact resolution and caching behavior also depends on the underlying networking environment.

Should I set connection lifetime equal to DNS TTL?

Not necessarily. DNS TTL and HTTP connection lifetime control different layers. The correct value depends on failover requirements, traffic patterns, infrastructure, and connection overhead.

Should every failed request be retried?

No. Retry only failures that are considered transient and safe to retry. Non-idempotent operations require additional safeguards.

Is DNS-aware failover enough for high availability?

No. It is one part of a resilient client architecture. Health checks, retry policies, timeouts, observability, load balancing, and safe deployment practices may also be required.

Conclusion

DNS-based failover looks simple from the infrastructure side:

Old IP
  ↓
New IP

But a long-running .NET application has connection pools that can continue using established connections after DNS records change.

That is why resilient HTTP clients need to consider both DNS resolution and connection lifetime.

A practical design combines:

DNS
 ↓
Connection Pool
 ↓
Connection Lifetime
 ↓
Timeouts
 ↓
Failure Detection
 ↓
Safe Retry
 ↓
Observability

The most important lesson is that DNS failover should be tested, not assumed.

For .NET 11 applications, measure how long your clients continue using old connections, how quickly they move to new endpoints, and what connection churn the configuration creates.

The right configuration is not the one with the shortest connection lifetime.

It is the one that provides predictable failover without turning normal HTTP traffic into a constant stream of new connections.