Introduction
Deploying your application to Azure Kubernetes Service (AKS) through an automated pipeline is a major achievement, but what happens after the deployment starts? As an engineer, you need to know how to inspect the cluster, verify that your application is healthy, and make it accessible from the internet.
This article provides a practical guide to using kubectl, the Kubernetes command-line tool, to monitor AKS deployments, retrieve public IP addresses, and troubleshoot common errors.
1. Connecting to Your AKS Cluster
Before you can query the cluster, you must authenticate your local terminal. Azure provides a command to download the cluster credentials and configure your local kubeconfig file.
Run the following command, replacing the resource group and cluster name with your actual values:
az aks get-credentials \
--resource-group linux-admin-course \
--name mudassar-kubi-courseOnce the command completes successfully, your local Kubernetes configuration contains the credentials needed to communicate with the AKS cluster. Subsequent kubectl commands will use the selected context.
To verify which cluster context is active, run:
kubectl config current-contextThis helps ensure that you are working with the intended cluster before running deployment or troubleshooting commands.
2. Verifying That Your Pods Are Running
Pods are the smallest deployable units in Kubernetes. They contain one or more containers that run your application.
To check whether your deployment was successful, list the pods in the current namespace:
kubectl get podsIf your application runs in a specific namespace, use:
kubectl get pods -n <namespace-name>Understanding Pod Statuses
When you run kubectl get pods, pay attention to the STATUS column.
Running: The pod is running, although this does not automatically mean that the application is healthy or ready to receive traffic.Pending: Kubernetes has not yet completed scheduling or preparing the pod. This may happen because of resource constraints or storage issues.ContainerCreating: Kubernetes is preparing the container. It may be downloading the image, setting up networking, or preparing storage.ImagePullBackOff: Kubernetes failed to pull the container image and is retrying with increasing delays.ErrImagePull: Kubernetes encountered an error while trying to download the image.CrashLoopBackOff: The container repeatedly starts and crashes, so Kubernetes delays subsequent restart attempts.
For a more detailed view, run:
kubectl get pods -o wideThis command also displays information such as the node running each pod and the pod's IP address.
Troubleshooting ImagePullBackOff or ErrImagePull
These errors indicate that Kubernetes cannot retrieve the container image.
Common causes include:
An incorrect image name or tag.
A container image that does not exist in Azure Container Registry (ACR).
Missing permissions for AKS to pull images from ACR.
Incorrect registry credentials or image pull secrets.
To investigate the issue, run:
kubectl describe pod <pod-name>Check the Events section near the bottom of the output. It often provides useful information about the failed image pull.
For example, a manifest-not-found error may indicate an incorrect image tag, while an authorization error may indicate a registry permission problem.
Verify the image name and tag in your deployment manifest. If the image is stored in ACR, also confirm that the AKS cluster has the required image-pull permissions.
Troubleshooting CrashLoopBackOff
CrashLoopBackOff means the container repeatedly starts and exits. Kubernetes attempts to restart it, but the failure continues.
Common causes include:
Missing environment variables.
Incorrect application configuration.
Missing secrets, such as database connection strings.
Application startup exceptions.
Memory exhaustion, which may result in an
OOMKilledtermination.
Start by checking the container logs:
kubectl logs <pod-name>If the pod contains multiple containers, specify the container name:
kubectl logs <pod-name> -c <container-name>If the container has already restarted, inspect the logs from its previous instance:
kubectl logs <pod-name> --previousYou can also inspect the pod details:
kubectl describe pod <pod-name>Review the container's last termination reason, restart count, and recent events. These details can help identify whether the problem comes from application code, configuration, or resource limits.
3. Finding Your Application's Public IP Address
Even when your pods are running, your application may not be accessible from the internet.
To expose an application externally, you typically configure a Kubernetes Service. If the Service uses the LoadBalancer type, AKS can provision an Azure Load Balancer and an external IP address, subject to your cluster configuration and Azure permissions.
To list your services, run:
kubectl get servicesFor a specific namespace, use:
kubectl get services -n <namespace-name>Understanding the Service Output
A typical output might look like this:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
my-aks-app-service LoadBalancer 10.0.120.45 20.50.60.70 80:31234/TCPHere is what the main columns mean:
NAME: The name of the Kubernetes Service.
TYPE: The Service type, such as ClusterIP, NodePort, or LoadBalancer.
CLUSTER-IP: The internal IP address used within the cluster.
EXTERNAL-IP: The externally assigned IP address, when available.
PORT(S): The ports exposed by the Service.
If the Service has type LoadBalancer, look at the EXTERNAL-IP column.
A valid IP address indicates that an external address has been assigned. You can try accessing the application through that address, provided the Service ports, application configuration, network rules, and health checks are correctly configured.
If the value is <pending>, Azure has not yet completed assigning the external address.
Troubleshooting a Stuck Pending State
External IP provisioning often takes a few minutes, but the exact time depends on the cluster and Azure infrastructure.
If the address remains <pending>, investigate the following areas.
1. Inspect the Service Events
Run:
kubectl describe service <service-name>Review the Events section for errors related to load balancer provisioning, IP allocation, or permissions.
2. Check Azure Permissions
AKS uses its configured identity to interact with Azure resources. If that identity lacks the required permissions, Azure may be unable to provision or update the load balancer resources.
Check the cluster identity configuration and the permissions associated with the relevant Azure resources.
3. Check Public IP Quotas
Your Azure subscription has limits on public IP address allocation. If the applicable quota is exhausted, creating another external IP address may fail.
Review the subscription's public IP resource usage and quota limits, and resolve any quota issue if necessary.
4. Verify the Service Configuration
Inspect the Service definition:
kubectl get service <service-name> -o yamlConfirm that the Service type is configured correctly and that its port mappings match the application requirements.
4. Checking Deployment Status
Checking pods is useful, but you should also verify whether Kubernetes has successfully rolled out your application.
List the deployments:
kubectl get deploymentsTo inspect a specific deployment, run:
kubectl describe deployment <deployment-name>You can also monitor the rollout:
kubectl rollout status deployment/<deployment-name>This command waits for the deployment rollout to complete and reports whether Kubernetes has successfully updated the deployment.
If the rollout fails or stalls, inspect the deployment and its pods to identify issues such as failed image pulls, unavailable replicas, or application startup failures.
5. Checking Application Logs and Events
Logs and Kubernetes events are essential when investigating deployment problems.
View Application Logs
kubectl logs <pod-name>To follow new log output as it appears:
kubectl logs -f <pod-name>For a deployment with multiple replicas, you can select pods using a label selector:
kubectl logs -l app=my-aks-app --all-containers=trueMake sure the selector matches the labels used by your deployment.
View Recent Cluster Events
To inspect events in the current namespace, run:
kubectl get events --sort-by=.metadata.creationTimestampEvents can reveal scheduling problems, image-pull failures, container restarts, and other issues that may not be obvious from the pod status alone.
Conclusion
Understanding kubectl get pods, kubectl get services, and kubectl rollout status helps you monitor AKS deployments and verify whether your application is running as expected. When problems occur, kubectl describe, kubectl logs, and Kubernetes events provide valuable diagnostic information.
The key is to check more than whether a pod is running. Verify that the deployment has completed, the containers are healthy, the Service is configured correctly, and the application is reachable through its intended network endpoint. These practices make it easier to troubleshoot failures and manage reliable cloud-native applications on Azure Kubernetes Service.
Join the conversation! Your thoughts help the community grow.