When users encounter 502 Bad Gateway and 503 Service Unavailable errors on a frontend application, panic often sets in. However, a systematic approach can quickly isolate the root cause.
A common starting point for incident triage is verifying cluster health—such as confirming that nodes and cluster memory are fully stable with no resource exhaustion. When infrastructure resources are healthy, the issue usually points directly to networking configurations, routing paths, API URL mappings, or backend application pods.
Understanding the Difference: 502 vs. 503
Before diving into commands, it helps to understand what the gateway or ingress controller is telling you:
503 Service Unavailable: The proxy (Ingress controller, API gateway) attempted to route a request, but found zero healthy backend endpoints available.
502 Bad Gateway: The proxy successfully reached a backend pod, but received an invalid, empty, or corrupted response (or the connection was abruptly refused/reset).
Step-by-Step Troubleshooting Guide
Step 1: Verify Cluster and Node Resources
As noted during initial triage, rule out infrastructure-level bottlenecks first:
Check node memory and CPU utilization to ensure pods aren't being OOM-killed (Out of Memory) or throttled.
Run
kubectl get nodesandkubectl describe node <node-name>to verify node conditions and resource capacity.
Step 2: Check Service Endpoints (Diagnosing 503 Errors)
If you are seeing 503 errors, your ingress controller cannot find active pods to hand traffic to.
Bash
kubectl get endpoints <your-backend-service-name>
If the endpoint list is empty (
<none>): Kubernetes has no healthy pods registered for this service.Common causes for empty endpoints:
Selector Mismatch: Labels on your deployment pods no longer match the selectors defined in your Kubernetes
Servicemanifest.Failed Readiness Probes: The pods are running, but failing their readiness checks, causing Kubernetes to automatically strip them from the endpoint pool. Inspect pods using
kubectl describe pod <pod-name>.
Step 3: Inspect Ingress Controller Logs (Diagnosing 502 Errors)
If you are seeing 502 errors, the request is hitting a backend pod, but failing during communication. Inspect the ingress controller logs (e.g., NGINX Ingress) to see what error the proxy returned:
Bash
kubectl logs -n ingress-nginx -l app.kubernetes.io/component=controller --tail=100
Look for messages like
upstream timed out,connection refused, orno live upstreams. These clues indicate whether the backend application crashed, took too long to reply, or is listening on the wrong port.
Step 4: Validate API URLs and Routing Configurations
When frontend applications throw these errors unpredictably, configuration drift in API routes is a frequent culprit:
Check Ingress Rules: Ensure that the paths defined in your Ingress resource correctly match the paths and ports expected by your backend services. A typo in a routing path will seamlessly drop traffic.
Verify Frontend Environment Variables: Confirm that the frontend application is pointing to the correct API URLs/endpoints. If a recent deployment or configuration update injected an outdated or malformed internal DNS name, the frontend will fail to communicate with backend microservices.
Network Policies: Ensure that strict
NetworkPoliciesaren't accidentally blocking traffic originating from the ingress controller pods to the backend API pods.
Summary Checklist for the Team
Confirm cluster node metrics and memory status are normal.
Run
kubectl get endpointsto check if backend services have active targets.Review Ingress controller logs to pinpoint whether it's a 502 (bad upstream response) or 503 (no upstreams found).
Double-check frontend environment variables and API URL configurations.
Join the conversation! Your thoughts help the community grow.