DNS is often treated as something an application can safely ignore once a hostname resolves successfully.
In production systems, that assumption can cause problems.
Applications depend on DNS for service discovery, database endpoints, regional failover, load balancing, and traffic routing. When infrastructure changes, DNS may return a different address, an endpoint may become unhealthy, or a previously resolved address may stop accepting connections.
A .NET application therefore needs to distinguish between two different questions:
Can DNS resolve this hostname?
and:
Can I actually reach a healthy service at the resolved endpoint?
Those are not the same thing.
A robust failover detection strategy combines DNS resolution with application-level health probes and connection validation. This article explains how to build that capability in .NET, how to avoid common DNS caching mistakes, and how to design detection logic that does not turn a temporary network problem into unnecessary failover activity.
DNS Resolution Is Not a Health Check
Consider an application connecting to:
api.example.internal
DNS may return:
10.10.1.20
The DNS lookup succeeds.
But the application behind that address could still be:
Therefore:
DNS success
≠
Application health
A useful failover architecture separates these concerns:
DNS
|
v
Resolve hostname
|
v
Candidate endpoint(s)
|
v
Network connectivity
|
v
Health probe
|
+----+----+
| |
Healthy Unhealthy
| |
Use Failover
What DNS Failover Actually Means
A typical DNS-based failover setup might look like:
api.example.internal
|
+---- Region A
| 10.10.1.20
|
+---- Region B
10.20.1.20
Under normal conditions:
Client
|
v
Region A
If Region A becomes unavailable, DNS infrastructure can eventually direct new resolutions toward Region B.
The challenge is that applications do not necessarily perform a fresh DNS lookup for every HTTP request.
DNS results can be cached at several layers:
Application
|
.NET networking stack
|
Operating system
|
DNS resolver
|
Authoritative DNS
This is why DNS failover is not necessarily instantaneous.
Build a Simple DNS Probe
For basic DNS validation, .NET provides Dns.GetHostAddressesAsync.
using System.Net;
var addresses =
await Dns.GetHostAddressesAsync("api.example.internal");
foreach (var address in addresses)
{
Console.WriteLine(address);
}
This answers:
Can the hostname currently be resolved?
It does not answer:
Is the service healthy?
That distinction should remain explicit in your monitoring design.
Resolve All Candidate Addresses
A hostname can resolve to multiple addresses.
For example:
api.example.internal
|
+-- 10.10.1.20
+-- 10.10.1.21
+-- 10.20.1.20
Do not automatically assume that the first returned address is the only endpoint.
A simple probe can inspect every returned address:
var addresses =
await Dns.GetHostAddressesAsync(hostname);
foreach (var address in addresses)
{
Console.WriteLine(
$"{hostname} -> {address}");
}
This is particularly important when DNS is being used for load distribution or regional failover.
DNS Resolution Should Be Timed
A health system should measure how long resolution takes.
var stopwatch = Stopwatch.StartNew();
var addresses =
await Dns.GetHostAddressesAsync(hostname);
stopwatch.Stop();
Console.WriteLine(
$"DNS lookup: {stopwatch.ElapsedMilliseconds} ms");
Track at least:
A sudden increase in DNS resolution time can be an early warning signal even if requests are still succeeding.
Add a Real Health Probe
After DNS resolution, perform an application-level health check.
For example:
using var client = new HttpClient
{
Timeout = TimeSpan.FromSeconds(5)
};
var response =
await client.GetAsync(
"https://api.example.internal/health");
Console.WriteLine(
$"Status: {(int)response.StatusCode}");
Now the monitoring process evaluates two independent signals:
DNS resolution
+
HTTP health
A healthy result might be:
DNS: Success
HTTP: 200
Health: Healthy
An unhealthy result might be:
DNS: Success
HTTP: Timeout
Health: Unhealthy
The second scenario is important because DNS can continue working while the service itself is unavailable.
Use a Dedicated Health Endpoint
A health endpoint should be lightweight.
For ASP.NET Core:
builder.Services.AddHealthChecks();
var app = builder.Build();
app.MapHealthChecks("/health");
app.Run();
The endpoint should answer a simple operational question.
For example:
HTTP 200
can indicate that the application is ready to receive traffic.
More advanced health checks can validate dependencies such as:
Database
Cache
Message broker
External API
Storage
But avoid making a basic liveness probe dependent on every external service.
A database outage should not necessarily cause a process-level liveness check to report that the entire application must be restarted.
Liveness and Readiness Are Different
A useful distinction is:
Liveness
Is the process alive?
Application process
|
v
Still functioning?
Readiness
Can the application currently handle traffic?
Application
|
+-- Database available?
+-- Required configuration loaded?
+-- Critical dependencies available?
|
v
Ready?
DNS failover should generally be based on the service's ability to handle traffic rather than simply whether its process exists.
Add Timeouts to Probes
A health probe without a timeout can become a monitoring problem itself.
For example:
using var cts =
new CancellationTokenSource(
TimeSpan.FromSeconds(3));
try
{
var response =
await client.GetAsync(
"https://api.example.internal/health",
cts.Token);
Console.WriteLine(response.StatusCode);
}
catch (OperationCanceledException)
{
Console.WriteLine("Health probe timed out.");
}
The timeout should be short enough to detect a genuine failure but long enough to avoid treating normal network variation as an outage.
Do Not Fail Over on One Bad Probe
A single failed request is not necessarily an outage.
Temporary causes include:
A better strategy uses consecutive failures.
For example:
Probe 1 -> Success
Probe 2 -> Success
Probe 3 -> Timeout
Probe 4 -> Success
should probably remain healthy.
Compare that with:
Probe 1 -> Timeout
Probe 2 -> Timeout
Probe 3 -> Timeout
Probe 4 -> Timeout
which provides stronger evidence of an actual problem.
Use Failure Thresholds
A simple detector can maintain consecutive failure counts.
int consecutiveFailures = 0;
if (healthy)
{
consecutiveFailures = 0;
}
else
{
consecutiveFailures++;
}
if (consecutiveFailures >= 3)
{
Console.WriteLine(
"Endpoint considered unhealthy.");
}
The threshold should be selected based on the application's recovery requirements.
For a critical service, the system may need faster detection.
For a service with naturally variable latency, a higher threshold can prevent false positives.
Add Recovery Thresholds Too
Failover systems need hysteresis.
Suppose the primary endpoint fails:
Primary
|
X
|
Failover
When the primary becomes healthy again, immediately switching traffic back can create instability.
Instead, require several successful probes:
Primary recovery:
Success
Success
Success
Success
|
v
Mark healthy
This is called a recovery threshold or recovery window.
It prevents a service that repeatedly alternates between healthy and unhealthy states from causing constant failover and failback.
Track the Resolved IP Address
A DNS failover detector should record the address returned by DNS.
For example:
var addresses =
await Dns.GetHostAddressesAsync(hostname);
foreach (var address in addresses)
{
Console.WriteLine(
$"{DateTimeOffset.UtcNow:o} {address}");
}
A monitoring system can then detect:
Previous:
10.10.1.20
Current:
10.20.1.20
This is useful because an address change can provide evidence that DNS failover has occurred.
However, an address change should not automatically be interpreted as a service failure.
DNS systems can legitimately return different addresses for load balancing.
Separate DNS Changes From Health Changes
A useful state model is:
DNS State
---------
Resolved
Changed
Failed
Health State
------------
Healthy
Degraded
Unhealthy
Unknown
This produces more useful diagnostics than a single Boolean.
For example:
DNS: Changed
Health: Healthy
could simply mean traffic distribution changed.
Another scenario:
DNS: Resolved
Health: Unhealthy
means the DNS infrastructure is functioning but the resolved service is not.
A third scenario:
DNS: Failed
Health: Unknown
means the monitoring system cannot determine the service state because name resolution itself failed.
Avoid Creating a New HttpClient Per Probe
This pattern is undesirable:
using var client = new HttpClient();
inside a frequently executed monitoring loop.
Repeated client creation can create unnecessary connection churn.
Instead, reuse HttpClient:
var client = new HttpClient
{
Timeout = TimeSpan.FromSeconds(5)
};
For larger applications, use IHttpClientFactory to manage client configuration and handlers.
For example:
builder.Services.AddHttpClient(
"HealthProbe",
client =>
{
client.Timeout =
TimeSpan.FromSeconds(5);
});
Then inject the configured client into the monitoring service.
DNS Refresh and Connection Reuse
There is an important interaction between DNS and long-lived HTTP connections.
Suppose:
api.example.internal
|
v
10.10.1.20
later changes to:
api.example.internal
|
v
10.20.1.20
An existing HTTP connection may still point to the old IP address.
DNS resolution and TCP connection lifetime are therefore separate concerns.
For HttpClient, connection lifetime can be controlled through SocketsHttpHandler.
For example:
var handler = new SocketsHttpHandler
{
PooledConnectionLifetime =
TimeSpan.FromMinutes(5)
};
var client = new HttpClient(handler);
This can allow connections to be periodically recycled so that new connections can use current DNS information.
The correct lifetime depends on the DNS TTL, failover requirements, traffic pattern, and infrastructure.
Do not choose an arbitrary value without considering those factors.
Health Probes Should Not Replace DNS Failover
A common architectural mistake is to assume that an application-level probe should modify DNS records directly.
In most architectures, these are separate responsibilities:
DNS / Traffic Management
|
| Decides where traffic should go
v
Application
|
| Reports health
v
Monitoring / Orchestration
The application can report that it is unhealthy.
A dedicated infrastructure layer can then decide whether DNS, load balancing, routing, or service discovery should change.
This separation reduces the risk of application code accidentally modifying global traffic configuration.
Build a Failover Detector
A simple detector can combine DNS and HTTP checks.
public sealed record EndpointHealth(
string Hostname,
IReadOnlyList<IPAddress> Addresses,
bool DnsHealthy,
bool ServiceHealthy,
TimeSpan DnsDuration,
TimeSpan ServiceDuration);
Then implement a probe:
public async Task<EndpointHealth> ProbeAsync(
string hostname,
HttpClient client,
CancellationToken cancellationToken)
{
var dnsTimer = Stopwatch.StartNew();
IPAddress[] addresses;
try
{
addresses =
await Dns.GetHostAddressesAsync(
hostname,
cancellationToken);
}
catch
{
dnsTimer.Stop();
return new EndpointHealth(
hostname,
[],
false,
false,
dnsTimer.Elapsed,
TimeSpan.Zero);
}
dnsTimer.Stop();
var serviceTimer = Stopwatch.StartNew();
try
{
using var response =
await client.GetAsync(
$"https://{hostname}/health",
cancellationToken);
serviceTimer.Stop();
return new EndpointHealth(
hostname,
addresses,
true,
response.IsSuccessStatusCode,
dnsTimer.Elapsed,
serviceTimer.Elapsed);
}
catch
{
serviceTimer.Stop();
return new EndpointHealth(
hostname,
addresses,
true,
false,
dnsTimer.Elapsed,
serviceTimer.Elapsed);
}
}
The important part is not the exact implementation.
It is the separation of:
DNS health
Service health
Timing
Resolved addresses
Add Observability
A failover detector is only useful if its results can be investigated.
Record metrics such as:
dns_resolution_success
dns_resolution_duration
dns_address_count
dns_address_changed
health_probe_success
health_probe_duration
health_probe_failures
consecutive_failures
consecutive_recoveries
Structured logs can also capture:
Hostname
Resolved IPs
Probe result
HTTP status
Failure reason
Detection timestamp
Recovery timestamp
Avoid logging sensitive request data simply because the probe system is collecting operational telemetry.
Test DNS Failover Before Production
Do not wait for a real regional outage to discover that failover detection does not work.
Create controlled tests.
For example:
Test 1: Healthy Primary
DNS -> Primary
Primary -> 200
Expected -> Healthy
Test 2: Primary Service Failure
DNS -> Primary
Primary -> Timeout
Expected -> Unhealthy
Test 3: DNS Address Change
DNS -> Secondary
Secondary -> 200
Expected -> Address changed + Healthy
Test 4: DNS Failure
DNS -> Error
Expected -> DNS unhealthy
Test 5: Recovery
Primary fails
|
v
Failover
|
v
Primary recovers
|
v
Multiple successful probes
|
v
Primary healthy
These scenarios should be automated where possible.
Benchmark Detection Latency
Failover detection itself should be measured.
A useful metric is:
Failure occurs
|
v
DNS/health probe detects failure
|
v
Failover decision
Measure:
Detection latency
Recovery latency
False-positive rate
Probe overhead
For example, if a service fails at 10:00:00 and the detector marks it unhealthy at 10:00:08:
Detection latency = 8 seconds
Do not optimize this number blindly.
Very aggressive probes can create more network traffic and increase the probability of reacting to transient failures.
Common Mistakes
Treating DNS Resolution as Health
A successful lookup only proves that DNS returned an answer.
Probing Only Once
One timeout is not enough evidence for many production systems.
Ignoring Recovery Hysteresis
Immediate failback can create repeated traffic switching.
Creating HttpClient Repeatedly
Reuse HTTP clients or use IHttpClientFactory.
Ignoring DNS Caching
Changing a DNS record does not necessarily mean every existing connection immediately moves to the new endpoint.
Using Extremely Short Timeouts
A timeout that is too aggressive creates false failures.
Making DNS Changes Directly From Application Code
Keep application health reporting separate from global traffic-management decisions.
Monitoring Only HTTP Status
A service returning HTTP 200 can still be operationally unhealthy if the health endpoint is poorly designed.
Troubleshooting Failover Detection
If DNS changes but traffic continues reaching the old endpoint, investigate:
DNS caching.
Existing TCP connections.
HttpClient connection pooling.
DNS TTL.
Resolver caching.
Reverse proxies.
Load balancers.
Service mesh behavior.
Application-level endpoint caching.
If DNS resolves correctly but the health probe fails, investigate:
Firewall rules.
TLS certificate validation.
Routing.
Security groups.
Service availability.
Health endpoint behavior.
Proxy configuration.
Network latency.
If failover happens too frequently, investigate:
Probe interval
Failure threshold
Recovery threshold
Timeout
Network variability
Health endpoint dependencies
The detector should be tuned using observed failure patterns rather than arbitrary values.
A Practical Production Checklist
Before using DNS-based failover detection in production, verify:
[ ] DNS resolution is measured separately from service health
[ ] Multiple resolved addresses are handled
[ ] Probe timeouts are configured
[ ] Consecutive failure thresholds are defined
[ ] Recovery thresholds are defined
[ ] DNS address changes are logged
[ ] HttpClient connections are reused
[ ] Connection lifetime is considered
[ ] Health endpoints are lightweight
[ ] Liveness and readiness are distinguished
[ ] Detection latency is measured
[ ] False positives are monitored
[ ] Failover scenarios are tested
[ ] Recovery scenarios are tested
[ ] Monitoring provides enough diagnostic context
Frequently Asked Questions
Does a DNS change immediately redirect existing .NET connections?
No. Existing TCP connections can continue to use the previously resolved address. DNS resolution and connection lifetime are separate mechanisms.
How often should a health probe run?
There is no universal interval. It should be based on the required detection time, network overhead, service characteristics, and acceptable false-positive rate.
Is a DNS health check enough for failover?
No. DNS availability does not prove application availability. Combine DNS resolution with an application-level health probe.
Should the application automatically switch to another IP?
That depends on the architecture. If DNS or a load balancer already provides failover, the application should generally respect that traffic-management layer instead of implementing a competing routing mechanism.
Does HttpClient always perform a DNS lookup for every request?
No. HTTP connection pooling means requests can reuse existing connections. DNS resolution and connection lifetime therefore need to be considered together when designing failover behavior.
Can DNS failover be tested locally?
Yes. Controlled DNS environments, test hostnames, container networks, or dedicated test infrastructure can be used to simulate address changes and service failures without modifying production DNS.
Conclusion
Reliable DNS failover detection requires more than checking whether a hostname resolves. DNS tells the application where traffic may go, while a health probe provides evidence about whether the resolved service can actually handle requests.
A robust .NET implementation should measure DNS resolution, track returned addresses, perform application-level health checks, use sensible timeouts and failure thresholds, and account for HTTP connection pooling when DNS changes occur. Recovery should be handled with the same care as failure so that a temporarily unstable endpoint does not cause repeated failover and failback.
The most useful design is one that treats DNS state and service health as separate signals and combines them into an observable failover decision. When those signals are measured consistently and tested under controlled failure scenarios, DNS-based failover becomes a predictable operational mechanism rather than something the application discovers only during an outage.
Join the conversation! Your thoughts help the community grow.