Containerized Apps: Auto Scaling & Monitoring with Kubernetes

namespace AutoScaleK8SDemo
{
    public class WeatherForecast
    {
        public DateTime Date { get; set; }
        public int TemperatureC { get; set; }
        public int TemperatureF => 32 + (int)(TemperatureC / 0.5556);
        public string? Summary { get; set; }
    }
}

using Microsoft.AspNetCore.Mvc;
namespace AutoScaleK8SDemo.Controllers
{
    [ApiController]
    [Route("[controller]")]
    public class WeatherForecastController : ControllerBase
    {
        private static readonly string[] Summaries = new[]
        {
        "Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching"
    };
        private readonly ILogger<WeatherForecastController> _logger;
        public WeatherForecastController(ILogger<WeatherForecastController> logger)
        {
            _logger = logger;
        }
        [HttpGet(Name = "GetWeatherForecast")]
        public IEnumerable<WeatherForecast> Get()
        {
            return Enumerable.Range(1, 5).Select(index => new WeatherForecast
            {
                Date = DateTime.Now.AddDays(index),
                TemperatureC = Random.Shared.Next(-20, 55),
                Summary = Summaries[Random.Shared.Next(Summaries.Length)]
            })
            .ToArray();
        }
    }
}

var builder = WebApplication.CreateBuilder(args);
// Add services to the container.
builder.Services.AddControllers();
// Learn more about configuring Swagger/OpenAPI at https://aka.ms/aspnetcore/swashbuckle
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();
var app = builder.Build();
// Configure the HTTP request pipeline.
app.UseSwagger();
app.UseSwaggerUI();
app.UseAuthorization();
app.MapControllers();
app.Run();

# Use the official .NET Core SDK as a parent image
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /app
# Copy the project file and restore any dependencies (use .csproj for the project name)
COPY *.csproj ./
RUN dotnet restore
# Copy the rest of the application code
COPY . .
# Publish the application
RUN dotnet publish -c Release -o out
# Build the runtime image
FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS runtime
WORKDIR /app
COPY --from=build /app/out ./
# Expose the port your application will run on
EXPOSE 80
# Start the application
ENTRYPOINT ["dotnet", "AutoScaleK8SDemo.dll"]

Step 4. Open the browser and hit the API URL to execute the endpoint.

Step 5. Create a deployment and service YAML file for Kubernetes to create deployments, pods, and services for our weather forecast service.

Deployment.YAML

apiVersion: apps/v1
kind: Deployment
metadata:
  name: weatherforecast-app-deployment  # Name of the deployment
spec:
  selector:
    matchLabels:
      app: weatherforecast-app  # Label selector to match pods controlled by this deployment
  template:
    metadata:
      labels:
        app: weatherforecast-app  # Labels applied to pods created by this deployment
    spec:
      containers:
        - name: weatherforecast-app  # Name of the container
          image: web-api:latest  # Docker image to use
          imagePullPolicy: Never
          ports:
          - containerPort: 80  # Port to expose within the pod
          resources:
            requests:
              memory: 20Mi
              cpu: "0.25"
            limits:
              memory: 400Mi
              cpu: "1"

Service.YAML

apiVersion: v1
kind: Service
metadata:
  name: weatherforecast-app-service  # Name of the service
spec:
  selector:
    app: weatherforecast-app  # Label selector to target pods with this label
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  type: NodePort  # Type of service (other options include ClusterIP, LoadBalancer, etc.)

Step 6. Apply deployment and service YAML files with kubectl commands.

kubectl apply -f deployment.yml
kubectl apply -f service.yml

Step 7. Check and verify deployment, instances, services, pods, logs, etc.

Step 8. Open the browser and hit the localhost URL with the Kubernetes service port, as shown in the above image.

Auto-Scaling Implementation with Kubernetes

First, we need a metric server for this functionality.

The Metric Server is like a traffic cop in a city full of roads (nodes) and cars (pods). It keeps an eye on how busy each road (node) is and how much space each car (pod) is taking up.

Following are a few points that help us understand why the metric server is important.

• Managing traffic: Just like a traffic cop helps manage traffic flow by monitoring how many cars are on the road, the Metric Server helps Kubernetes manage the workload by keeping track of how busy each part of the system is.
• Making decisions: When roads get too crowded, the traffic cop might direct some cars to take a different route. Similarly, Kubernetes uses the information from the Metric Server to decide if it needs to add more resources (like more roads) or move some workloads around to keep everything running smoothly.
• Keeping things running smoothly: By knowing how much space each car (pod) is taking up on the road (node), Kubernetes can ensure that resources are used efficiently. It prevents situations where some parts of the system have too much traffic while others are empty.
• Planning ahead: Just like city planners use traffic data to decide where to build new roads, Kubernetes administrators can use data from the Metric Server to plan for future needs. They can see how much traffic the system is handling and make decisions about adding more resources if needed.

Let’s start the configuration

Step 1. Download the metric server file from the below path.

https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Step 2. Modify the server argument to add - --kubelet-insecure-tls

• The Metrics Server talks to the Kubelets to get information about how your cluster is doing.
• Usually, this communication is kept safe using TLS, a kind of security measure.
• But in a setup like Docker Desktop for local development, the Kubelets might use certificates that aren't automatically trusted. This can cause security issues.
• To work around this in development, you can use a flag called --kubelet-insecure-tls. It tells the Metrics Server to not worry about checking the certificates when it talks to the Kubelets.
• But remember, while this is handy for development, it's not safe for production because it opens up security risks.

Step 3. Apply the components.yaml file with the help of Kubectl to configure the metric server on the cluster.

kubectl apply -f components.yaml

Step 4. Create a new hpa.yaml file for horizontal pod scaling.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: weatherforecast-hpa
spec:
  minReplicas: 1
  maxReplicas: 5
  metrics:
    - resource:
        name: cpu
        target:
          averageUtilization: 40
          type: Utilization
      type: Resource
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: weatherforecast-app-deployment

Explanation

• API version: Specifies the Kubernetes API version being used. In this case, it's autoscaling/v2, indicating that it's using the v2 version of the Autoscaling API.
• Kind: specifies the kind of Kubernetes resource being defined. Here, it's HorizontalPodAutoscaler, indicating that it's an HPA resource.
• Metadata: Contains metadata about the HPA resource, including its name.
• Spec: defines the specification of the HPA, including its behavior.
• Min replicas: Specifies the minimum number of replicas that should be running at any given time. In this case, it's set to 1, meaning there should always be at least one replica running.
• Max replicas: Specifies the maximum number of replicas that can be scaled up to. In this case, it's set to 5, meaning the auto scaler can create up to 5 replicas if necessary.
• Metrics: Specifies the metrics used for autoscaling.
• Resource: Indicates that the auto scale should scale based on resource metrics such as CPU or memory usage.
• Name: Specifies the name of the resource metric being used. Here, it's CPU, indicating that the autoscaler will scale based on CPU usage.
• Target: Specifies the target value for the metric.
• Average utilization: specifies the target average utilization of the resource metric. In this case, it's set to 40, meaning the auto scaler will try to maintain an average CPU utilization of 40%.
• Type: Specifies the type of target value. Here, it's Utilization.
• Type: Specifies the type of metric being used for autoscaling. Here, it's Resource.
• Scale target ref: Specifies the reference to the resource that should be scaled by this HPA.
• API version: Specifies the API version of the resource being scaled. Here, it's apps/v1, indicating an API version of Kubernetes apps.
• Kind: Specifies the kind of resource being scaled. Here, it's Deployment.
• Name: Specifies the name of the deployment being scaled. In this case, it's weather forecast-app-deployment.

Step 5. Apply hpa YAML file with the kubectl command.

kubectl apply -f hpa.yaml

Step 6. By default, vertical pod auto-scaling is not available in Kubernetes; for that, we need to install the same.

https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler

Step 7. Go to the autoscaler\vertical-pod-autoscaler\hack path and execute the vpa-up file, then it will install vpa.

Step 8. Create a new vpa.yml file for vertical pod scaling

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: weatherforecast-app-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: weatherforecast-app-deployment
  updatePolicy:
    updateMode: "Off"

Explanation

• API version: This indicates the version of the Kubernetes API being used, specifically the Autoscaling API version 1 (autoscaling.k8s.io/v1).
• Kind: Specifies the kind of Kubernetes resource being defined, which in this case is a VerticalPodAutoscaler.
• Metadata: Contains information about the VPA, including its name.
• Spec: defines the specifications of the VPA, including its behavior.
• Target ref: Specifies the reference to the resource that should be adjusted by this VPA. In this case, it's a deployment named "weather forecast-app-deployment".
• Update policy: Specifies the policy for updating the pods managed by the VPA.
• Update mode: Indicates the update mode for the VPA. Here, it's set to "Off", which means the VPA is not actively updating the resource. This likely implies that the VPA is being used in an advisory mode, providing recommendations for resource requests and limits rather than directly adjusting them.

There are the following types of update modes in vertical pod scaling

Auto Mode

In Auto mode, the VPA automatically adjusts the resource requests and limits for your application's pods based on their actual usage. If a pod needs more resources to handle its workload, the VPA will increase its resource allocations. Similarly, if a pod is using fewer resources, the VPA will decrease its allocations. It's like having a smart system that dynamically adjusts the resources your application pods need to run efficiently.

Recreate Mode

In Recreate mode, the VPA recalculates the resource requirements for your pods based on their usage patterns, but it requires the pods to be recreated with the updated configurations. This means that when the VPA determines that a pod needs more or fewer resources, it will delete the existing pod and create a new one with the updated resource allocations. During this process, there may be a brief period of downtime as the new pod is created.

Initial Mode

In Initial mode, the VPA uses historical usage data to compute resource recommendations for your pods, but it doesn't apply these recommendations immediately. Instead, the recommendations serve as starting points when the pods are initially created or scaled up. This mode helps ensure that new pods are provisioned with appropriate resource allocations from the beginning, based on past usage patterns.

Off Mode

In off mode, the VPA doesn't actively adjust the resource requests and limits for your pods. It simply observes and collects data on their resource usage without making any changes. This mode is useful when you want to monitor your application's resource usage trends without allowing the VPA to modify the pod configurations. It's like having a monitoring tool that watches your pods but doesn't interfere with their resource allocations.

Step 9. Apply vpa YAML file to the cluster with the help of kubectl.

kubectl apply -f vpa.yaml

Monitor Applications with High Traffic

Let’s generate a load on the service that we hosted before

Step 1. Execute the below command to create a load.

kubectl run -i --tty load-generator --rm --image=busybox:1.28 --restart=Never -- /bin/sh -c "while sleep 0.01; do wget -q -O- http://weatherforecast-app-service/WeatherForecast; done"

Note. If you want more traffic, then run multiple instances of the load generator at different prompts.

As we can see, the load is starting to generate in the above image.

As you can see in the above image the initial target is low and the number of replicas is also minimal, But our load generator starts creating loads, and due to that, our replicas start increasing, which means Kubernetes creates new pods to handle traffic with the help of auto-scaling.

Also, once we stopped the load generator, our number of replicas decreased, as shown in the below images

GitHub

https://github.com/Jaydeep-007/AutoScaleK8SDemo/tree/master

Conclusion

In this article, we discussed the basics of Docker and Kubernetes with auto-scaling. Also, step-by-step implementation of the Weather Forecast API. Later on, we containerize the service, apply the auto-scale configuration, and monitor the application with high traffic.