Automating DevOps Tasks with Python: Building an Infrastructure Health Checker

As infrastructure grows, checking application endpoints, network ports and system health manually can quickly become repetitive. This was one of the areas where I started using Python more actively alongside my DevOps work.

Automation does not always need to begin with a large platform or complex framework. Sometimes a small utility that removes repetitive checks can provide immediate value.

In this article, I will build a simple Infrastructure Health Checker using Python. The tool will check application endpoints, network connectivity, CPU usage, memory usage and disk utilisation, then present the results in a clear format.

This is also a useful example of how software development skills can complement infrastructure and DevOps engineering.

What We Are Building

Python Health Checker
↓
HTTP Endpoint Checks
+
TCP Port Checks
+
CPU, Memory and Disk Checks
↓
Health Summary

Why Build a Health Checker?

Imagine an engineer receives an alert saying that an application may be unavailable.

The first few checks may include:

Is the application endpoint responding?
Is the required network port reachable?
Is CPU utilisation unusually high?
Is memory usage approaching a limit?
Is the filesystem running out of space?

Each check is simple individually.

Repeating the same checks manually across multiple systems is where automation becomes useful.

Manual Checks
↓
Repeated Commands
↓
More Time + Inconsistent Results

Python Automation
↓
Consistent Checks
↓
Faster Health Summary

Project Structure

I will keep the project intentionally simple.

infrastructure-health-checker/
│
├── health_checker.py
├── requirements.txt
└── config.py

The main application logic will live inside health_checker.py.

Installing the Required Python Packages

For this example, I will use two commonly used Python libraries:

requests for HTTP endpoint checks
psutil for local system resource information

The requirements.txt file can contain:

requests
psutil

Install them using:

pip install -r requirements.txt

Defining the Systems to Check

Instead of hardcoding every target inside the application logic, I prefer keeping environment configuration separate.

Inside config.py:

HTTP_ENDPOINTS = [
    "https://example.com/health",
    "https://api.example.com/health"
]

TCP_TARGETS = [
    {
        "host": "example.com",
        "port": 443
    },
    {
        "host": "api.example.com",
        "port": 443
    }
]

CPU_WARNING_THRESHOLD = 80
MEMORY_WARNING_THRESHOLD = 80
DISK_WARNING_THRESHOLD = 85

These values are examples only. In a real environment, the targets would represent systems that the team is authorised to monitor.

Checking an HTTP Endpoint

The first function checks whether an HTTP endpoint responds successfully.

import requests


def check_http_endpoint(url):
    try:
        response = requests.get(url, timeout=5)

        if response.ok:
            return {
                "target": url,
                "status": "HEALTHY",
                "details": f"HTTP {response.status_code}"
            }

        return {
            "target": url,
            "status": "UNHEALTHY",
            "details": f"HTTP {response.status_code}"
        }

    except requests.RequestException as error:
        return {
            "target": url,
            "status": "UNHEALTHY",
            "details": str(error)
        }

There are two things I consider important here.

Timeout

The request uses:

timeout=5

Without a timeout, an automation script may wait much longer than expected when a service is not responding correctly.

Exception Handling

Network failures should not crash the entire script.

Instead, the exception is captured and returned as part of the health result.

Checking Network Port Connectivity

An application may be running while another required service is unreachable.

Python's built-in socket module can perform a simple TCP connection check.

import socket


def check_tcp_port(host, port):
    try:
        with socket.create_connection((host, port), timeout=5):
            return {
                "target": f"{host}:{port}",
                "status": "HEALTHY",
                "details": "TCP connection successful"
            }

    except (socket.timeout, socket.gaierror, OSError) as error:
        return {
            "target": f"{host}:{port}",
            "status": "UNHEALTHY",
            "details": str(error)
        }

This does not prove that the application protocol itself is functioning correctly.

It confirms that a TCP connection to the target port can be established from the machine running the health checker.

Important: port checks should only be performed against systems you own, manage or are authorised to test.

Checking CPU Usage

Next, I want the tool to identify unusually high CPU usage on the system where it is running.

import psutil


def check_cpu(threshold):
    cpu_usage = psutil.cpu_percent(interval=1)

    status = "HEALTHY"

    if cpu_usage >= threshold:
        status = "WARNING"

    return {
        "target": "CPU",
        "status": status,
        "details": f"{cpu_usage}% used"
    }

If CPU usage is greater than or equal to the configured threshold, the function returns a warning.

A single CPU sample should not normally be treated as proof of an incident. In production monitoring, I would evaluate sustained utilisation over time.

Checking Memory Usage

The same approach can be used for memory.

def check_memory(threshold):
    memory = psutil.virtual_memory()

    status = "HEALTHY"

    if memory.percent >= threshold:
        status = "WARNING"

    return {
        "target": "Memory",
        "status": status,
        "details": f"{memory.percent}% used"
    }

This gives us a quick indication of whether memory utilisation has crossed our configured threshold.

Checking Disk Usage

Disk capacity is another common operational check.

def check_disk(path, threshold):
    disk = psutil.disk_usage(path)

    status = "HEALTHY"

    if disk.percent >= threshold:
        status = "WARNING"

    return {
        "target": f"Disk {path}",
        "status": status,
        "details": f"{disk.percent}% used"
    }

On a Linux server, I might check:

Additional mount points could also be checked depending on the application architecture.

Building the Complete Health Checker

Now I can combine the individual checks into one application.

import socket

import psutil
import requests

from config import (
    HTTP_ENDPOINTS,
    TCP_TARGETS,
    CPU_WARNING_THRESHOLD,
    MEMORY_WARNING_THRESHOLD,
    DISK_WARNING_THRESHOLD
)


def check_http_endpoint(url):
    try:
        response = requests.get(url, timeout=5)

        if response.ok:
            return {
                "target": url,
                "status": "HEALTHY",
                "details": f"HTTP {response.status_code}"
            }

        return {
            "target": url,
            "status": "UNHEALTHY",
            "details": f"HTTP {response.status_code}"
        }

    except requests.RequestException as error:
        return {
            "target": url,
            "status": "UNHEALTHY",
            "details": str(error)
        }


def check_tcp_port(host, port):
    try:
        with socket.create_connection((host, port), timeout=5):
            return {
                "target": f"{host}:{port}",
                "status": "HEALTHY",
                "details": "TCP connection successful"
            }

    except (socket.timeout, socket.gaierror, OSError) as error:
        return {
            "target": f"{host}:{port}",
            "status": "UNHEALTHY",
            "details": str(error)
        }


def check_cpu(threshold):
    cpu_usage = psutil.cpu_percent(interval=1)

    status = "HEALTHY"

    if cpu_usage >= threshold:
        status = "WARNING"

    return {
        "target": "CPU",
        "status": status,
        "details": f"{cpu_usage}% used"
    }


def check_memory(threshold):
    memory = psutil.virtual_memory()

    status = "HEALTHY"

    if memory.percent >= threshold:
        status = "WARNING"

    return {
        "target": "Memory",
        "status": status,
        "details": f"{memory.percent}% used"
    }


def check_disk(path, threshold):
    disk = psutil.disk_usage(path)

    status = "HEALTHY"

    if disk.percent >= threshold:
        status = "WARNING"

    return {
        "target": f"Disk {path}",
        "status": status,
        "details": f"{disk.percent}% used"
    }


def print_result(result):
    print(
        f"[{result['status']}] "
        f"{result['target']} - "
        f"{result['details']}"
    )


def main():
    results = []

    for endpoint in HTTP_ENDPOINTS:
        results.append(
            check_http_endpoint(endpoint)
        )

    for target in TCP_TARGETS:
        results.append(
            check_tcp_port(
                target["host"],
                target["port"]
            )
        )

    results.append(
        check_cpu(CPU_WARNING_THRESHOLD)
    )

    results.append(
        check_memory(MEMORY_WARNING_THRESHOLD)
    )

    results.append(
        check_disk("/", DISK_WARNING_THRESHOLD)
    )

    print("\nInfrastructure Health Check")
    print("=" * 60)

    for result in results:
        print_result(result)

    unhealthy = [
        result
        for result in results
        if result["status"] == "UNHEALTHY"
    ]

    warnings = [
        result
        for result in results
        if result["status"] == "WARNING"
    ]

    print("=" * 60)

    if unhealthy:
        print(
            f"Overall Status: UNHEALTHY "
            f"({len(unhealthy)} failed check(s))"
        )
        return 2

    if warnings:
        print(
            f"Overall Status: WARNING "
            f"({len(warnings)} warning(s))"
        )
        return 1

    print("Overall Status: HEALTHY")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Understanding the Main Function

The main() function acts as the coordinator for the application.

It runs each health check and stores the result inside a list.

main()
↓
Check HTTP Endpoints
↓
Check TCP Ports
↓
Check CPU
↓
Check Memory
↓
Check Disk
↓
Generate Overall Status

Using Exit Codes

One small feature that makes the script more useful for DevOps automation is returning different process exit codes.

In this example:

0 means healthy
1 means warning
2 means unhealthy

This allows another automation system to use the result.

For example, a CI/CD pipeline could fail when the health checker returns a non-zero exit code.

Example Output

A successful run may look like this:

Infrastructure Health Check
============================================================

[HEALTHY] https://example.com/health - HTTP 200
[HEALTHY] https://api.example.com/health - HTTP 200
[HEALTHY] example.com:443 - TCP connection successful
[HEALTHY] api.example.com:443 - TCP connection successful
[HEALTHY] CPU - 21.4% used
[HEALTHY] Memory - 54.8% used
[HEALTHY] Disk / - 62.1% used

============================================================
Overall Status: HEALTHY

If memory usage is above the warning threshold:

[WARNING] Memory - 87.3% used

Overall Status: WARNING (1 warning(s))

Adding Structured Logging

Printing information to the terminal is useful during development, but automation becomes easier to operate when results are structured and logged consistently.

Python's built-in logging module can be used instead of relying only on print().

import logging


logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)s %(message)s"
)

logger = logging.getLogger(__name__)


logger.info("Starting infrastructure health check")

This makes timestamps and severity levels available in the output.

Later, those logs could be forwarded into an observability platform.

Keeping Configuration Outside the Code

As the tool grows, I would avoid embedding environment-specific values inside the Python code.

Configuration could eventually come from:

Environment variables
YAML configuration
JSON configuration
Command-line arguments
A configuration service

For example:

import os


cpu_threshold = int(
    os.getenv(
        "CPU_WARNING_THRESHOLD",
        "80"
    )
)

This allows the same application to behave differently across environments without changing the source code.

Containerising the Health Checker

Because the tool is written in Python, it can also be packaged as a Docker container.

FROM python:3.12-slim

WORKDIR /app

COPY requirements.txt .

RUN pip install \
    --no-cache-dir \
    -r requirements.txt

COPY . .

CMD ["python", "health_checker.py"]

The image can then be built using:

docker build -t infrastructure-health-checker:1.0 .

Running the Checker on a Schedule

Once the health checker is automated, it could be executed periodically.

For example, a Linux cron job could run it every five minutes:

*/5 * * * * /usr/bin/python3 /opt/health-checker/health_checker.py

The same idea could later be implemented using Kubernetes CronJobs or an automation platform.

Integrating the Checker with CI/CD

The health checker can also become part of a deployment pipeline.

A deployment process could look like:

Build Application
↓
Run Tests
↓
Deploy Application
↓
Run Python Health Checker
↓
Validate Application Health
↓
Mark Deployment Successful

This connects Python development directly with DevOps automation.

Where I Would Take This Next

The application in this article is intentionally small, but it creates a foundation for more advanced automation.

Some improvements I would consider include:

Checking multiple servers concurrently
Returning results as JSON
Storing historical health data
Sending notifications when checks fail
Adding Kubernetes API checks
Checking certificate expiry
Checking database connectivity
Providing a small REST API
Creating a simple health dashboard
Adding automated diagnostic suggestions

At that point, the project starts becoming more than a script. It begins to behave like a small operational tool.

Moving from Scripts to Engineering

One lesson I learned while working with automation is that there is an important difference between writing a script and building a reliable tool.

A small script may solve an immediate problem.

A reusable engineering tool should also consider:

Error handling
Timeouts
Configuration management
Logging
Exit codes
Testing
Security
Maintainability
Deployment

Thinking about these areas helped me start approaching DevOps automation from more of a software engineering perspective.

Final Architecture

Infrastructure Health Checker
↓
Python Application
↓
HTTP Checks + TCP Checks + Resource Checks
↓
Result Evaluation
↓
HEALTHY / WARNING / UNHEALTHY
↓
Logs + Exit Code
↓
CI/CD or Scheduled Automation

Conclusion

In this article, I used Python to automate several common infrastructure health checks.

The tool can:

Check HTTP endpoints
Test TCP connectivity
Monitor CPU utilisation
Monitor memory utilisation
Check disk capacity
Handle failures without crashing
Generate an overall health status
Return meaningful process exit codes
Integrate with CI/CD automation

The most useful part of this exercise for me was not simply automating commands.

It was learning to take an operational problem and turn it into a small, reusable piece of software.

This is where I started seeing a stronger connection between DevOps and development. Infrastructure engineers increasingly need to understand not only how systems operate, but also how to build tools that make those systems easier to operate.

My next step: After using Python to automate operational checks, I became interested in whether intelligent analysis could help engineers understand large volumes of logs faster. In the next article, I will explore how AI can be used to analyse DevOps logs and identify potential operational issues while keeping engineering decisions under human control.