One of the confusing situations in Kubernetes is when all application Pods show a Running status, but users still cannot access the application. The Deployment appears healthy, the containers have started, and there are no obvious restart errors. However, requests to the application return connection errors, time out, or never reach the expected service.
This happens because a running container does not necessarily mean that an application is ready to receive traffic. Kubernetes manages several layers between a client and an application, including Pods, Services, EndpointSlices, DNS, network policies, and ingress controllers. A problem in any of these layers can make an otherwise healthy application inaccessible.
For example, an ASP.NET Core application might start successfully inside a container but listen on a different port than the Kubernetes Service expects. Another possibility is that the Service selector does not match the Pod labels. The Pods are running, but the Service has no suitable backends.
Instead of restarting Pods or changing multiple configuration files, troubleshoot the connection one layer at a time. This article explains how to identify where traffic is being interrupted and how to restore access without making unnecessary changes to a working deployment.
Running and Ready Are Not the Same Thing
Start by understanding the difference between a Pod's Running status and its readiness condition.
The Running status generally means that the Pod has been bound to a node and its containers have been created and started. It does not guarantee that the application inside those containers is functioning correctly or accepting requests.
Readiness is a separate condition. Kubernetes uses readiness probes to determine whether a Pod should receive traffic from Services that select it.
For example, check the Pods using:
kubectl get pods
You might see:
NAME READY STATUS RESTARTS AGE
my-api-7d8f9c6d4a-abc12 1/1 Running 0 10m
my-api-7d8f9c6d4a-def34 1/1 Running 0 10m
Both Pods are running and ready. If users still cannot access the application, the problem may be outside the container's immediate health.
Now inspect the Pod details:
kubectl describe pod my-api-7d8f9c6d4a-abc12
Review the Conditions and Events sections. Check whether the Ready condition is true and whether the readiness probe has reported any failures.
If the Pod is running but not ready, Kubernetes may exclude it from the Service's ready endpoints. If it is ready, continue investigating the Service and network path.
This distinction prevents a common troubleshooting mistake: treating every connectivity issue as a container startup problem.
Check Whether the Service Selects the Correct Pods
A Kubernetes Service provides a stable way to access a group of Pods. It uses label selectors to identify which Pods should receive traffic.
Consider a Deployment with the following labels:
metadata:
labels:
app: my-api
spec:
template:
metadata:
labels:
app: my-api
The corresponding Service should use a matching selector:
apiVersion: v1
kind: Service
metadata:
name: my-api-service
spec:
selector:
app: my-api
ports:
- name: http
port: 80
targetPort: 8080
Here, the Service selects Pods with the label app=my-api.
If the Service instead uses app=api, it will not select these Pods. The application containers may be running perfectly, but the Service will have no matching backends.
Inspect the Service:
kubectl describe service my-api-service
You can also retrieve its selector:
kubectl get service my-api-service -o yaml
Then list the Pods using the same selector:
kubectl get pods -l app=my-api
Compare the labels carefully. A small naming difference is enough to prevent the Service from selecting the intended Pods.
When troubleshooting a Service with no endpoints, check the selector before changing network settings or restarting the application.
Verify the Service Ports and Container Listening Port
A port mismatch is another common reason an application becomes inaccessible.
A Kubernetes Service defines a port that clients use and a targetPort that identifies the port on the selected Pods.
For example:
ports:
- port: 80
targetPort: 8080
This means clients connect to port 80 on the Service, while Kubernetes forwards traffic to port 8080 on the selected Pod.
The targetPort must match the port on which the application actually listens.
Consider an ASP.NET Core application configured to listen on port 8080. If the Service forwards traffic to port 5000, requests will not reach the application correctly.
Check the application configuration, container environment variables, and container startup arguments. Do not assume that the port declared in the Dockerfile or the containerPort field automatically configures the application to listen on that port.
The containerPort field in a Pod specification is useful documentation and can be referenced by a named Service targetPort, but it does not itself make an application listen on that port.
You can inspect the Service configuration with:
kubectl get service my-api-service -o yaml
If you use a named targetPort, verify that the container declares the same port name.
A clear port configuration helps prevent mistakes when the application port changes between environments.
Make Sure the Application Listens on the Correct Network Interface
An application can listen on the correct port and still be inaccessible from outside its own container.
This commonly happens when the application listens only on the loopback interface, such as 127.0.0.1.
The loopback interface allows a process to communicate with services inside its own network namespace. It does not make the application available through the Pod's network interface.
For an ASP.NET Core application, verify the configured listening address.
For example, an application can be configured through an environment variable:
ENV ASPNETCORE_URLS=http://+:8080
The plus sign tells ASP.NET Core to listen on all available network interfaces for the specified port.
You can also configure the address through application startup settings or command-line arguments.
The important point is that the application must listen on an address reachable through the Pod network, not only on localhost.
When investigating this issue, inspect the application logs and, if the container includes suitable diagnostic tools, check the listening sockets.
For example:
kubectl exec -it my-api-pod -- sh
Inside the container, use an available networking utility to inspect the listening port. Minimal production images may not contain tools such as curl, netstat, or ss, so a temporary diagnostic container may be more appropriate.
Inspect EndpointSlices When the Service Has No Backends
Kubernetes uses EndpointSlices to track the network endpoints associated with a Service.
If the Service selector is incorrect, the selected Pods are not ready, or another condition prevents them from becoming usable endpoints, the Service may not have the expected backend addresses.
Check the EndpointSlices:
kubectl get endpointslices \
-l kubernetes.io/service-name=my-api-service
Inspect the details:
kubectl describe endpointslice my-api-service-xxxxx
The exact EndpointSlice name will differ.
Check whether the expected Pod IP addresses and ports are present. Also review endpoint conditions, particularly readiness.
If the Service has no usable endpoints, return to the Pod labels, readiness conditions, and Service selector. If the endpoints exist but requests still fail, investigate the network path.
This step is useful because it separates two different problems: a Service that has no suitable backend and a Service that has backends but cannot deliver traffic to them.
Test Connectivity From Inside the Cluster
Testing the application only from a developer's machine does not reveal every network problem.
A request may fail because of an ingress configuration, firewall rule, DNS problem, or network policy. Testing from another Pod helps determine whether the application is reachable inside the cluster.
You can create a temporary diagnostic Pod:
kubectl run network-test \
--rm -it \
--restart=Never \
--image=curlimages/curl \
-- sh
From inside the diagnostic container, test the Service:
curl -v http://my-api-service
If the Service listens on a non-default port, include it in the URL:
curl -v http://my-api-service:80
You can also test the Pod IP and application port directly:
curl -v http://10.244.0.15:8080
Replace the example address with the actual Pod IP and port.
The results help narrow down the problem.
If direct Pod access works but Service access fails, investigate the Service selector, EndpointSlices, port mapping, and cluster networking.
If direct Pod access also fails, check whether the application is listening correctly and whether network policies or other controls are blocking the connection.
If internal Service access works but external access fails, focus on the ingress controller, load balancer, DNS, firewall, and external routing configuration.
Review Network Policies
Kubernetes NetworkPolicies can control traffic entering and leaving Pods. A policy may allow the application to run normally while preventing clients from reaching it.
For example, a namespace may have a default-deny ingress policy:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-ingress
namespace: production
spec:
podSelector: {}
policyTypes:
- Ingress
This policy selects all Pods in the namespace and denies ingress traffic unless another applicable policy allows it.
If the application requires traffic from an ingress controller or another service, an appropriate allow policy must exist.
Inspect the policies in the namespace:
kubectl get networkpolicies -n production
Then review the relevant policy:
kubectl describe networkpolicy my-api-policy -n production
Check the Pod selectors, namespace selectors, ports, and allowed sources.
Remember that NetworkPolicy enforcement depends on the cluster networking implementation. A policy object existing in the cluster does not necessarily prove that the networking layer is enforcing it correctly.
Avoid deleting a restrictive policy simply to test connectivity in production. First identify the intended traffic sources and create a narrowly scoped rule that permits only the required communication.
Troubleshoot Ingress and DNS Configuration
If the application works through its ClusterIP Service but users cannot reach it through a public hostname, the problem may be in the ingress layer.
Start by checking the Ingress resource:
kubectl get ingress
Inspect its configuration:
kubectl describe ingress my-api-ingress
Verify the hostname, path rules, backend Service name, backend port, and ingress class.
A common mistake is configuring an Ingress to forward traffic to a Service port that does not exist.
For example, the Service might expose port 80, but the Ingress references port 8080. The application may work internally through the Service while the ingress route fails.
Also verify that the ingress controller is running and that its logs do not report backend or routing errors.
DNS should be checked separately. Confirm that the application's hostname resolves to the expected external load balancer or ingress address.
For internal service discovery, Kubernetes DNS normally allows services to be addressed by their names within the cluster. If a service is in another namespace, its fully qualified service name can be used.
For example:
my-api-service.production.svc.cluster.local
If DNS resolution fails, inspect the CoreDNS Pods and their logs rather than changing the application deployment immediately.
Common Mistakes During Connectivity Troubleshooting
Mistake | Why it causes problems | Better approach |
|---|
Assuming Running means available | The application may not be ready to receive traffic | Check the Ready condition |
Restarting Pods repeatedly | Does not correct Service or network configuration | Identify the failing layer |
Ignoring Service selectors | The Service may have no matching Pods | Compare selectors with Pod labels |
Assuming containerPort configures the application | It does not make the process listen on that port | Verify the actual listening port |
Testing only from outside the cluster | Does not isolate internal connectivity problems | Test from a diagnostic Pod |
Removing NetworkPolicies to restore access | Can expose workloads unnecessarily | Add narrowly scoped allow rules |
Changing Ingress before checking the Service | The problem may be inside the cluster | Test Pod and Service connectivity first |
Best Practices for Reliable Kubernetes Networking
A reliable application deployment should include network and service checks as part of its normal operational process.
Configure readiness probes that represent the application's ability to receive requests.
Keep Service selectors consistent with Deployment Pod labels.
Verify that Service targetPort values match the application's actual listening port.
Configure applications to listen on an address reachable through the Pod network.
Inspect EndpointSlices when a Service does not route traffic as expected.
Use diagnostic Pods to test connectivity from inside the cluster.
Apply NetworkPolicies that allow only the required application traffic.
Validate Ingress backend services, ports, paths, and hostnames.
Monitor DNS and ingress controller health.
Include an end-to-end connectivity check in deployment verification.
These checks make troubleshooting more predictable. They also help identify configuration problems before they affect users.
Summary
Kubernetes Pods can show a Running status while an application remains unavailable because container status is only one part of the overall request path.
The problem may be caused by an unsuccessful readiness check, incorrect Service selector, port mismatch, application listening address, missing EndpointSlices, restrictive NetworkPolicy, or incorrect Ingress and DNS configuration.
The most effective troubleshooting approach is to move through the network path systematically. Start with the Pod, verify the application is listening, inspect the Service and its endpoints, test connectivity from inside the cluster, and then investigate ingress and external routing.
Avoid restarting healthy Pods or removing security policies before identifying the actual problem. A structured investigation helps restore access faster and prevents unnecessary changes to a working Kubernetes environment.
Join the conversation! Your thoughts help the community grow.