Cloud  

Argo Workflows with GitHub Actions From Scratch

ArgoCD

Introduction

Argo Workflows is a Kubernetes-native workflow engine that helps to automate complex workflows in a cloud-native environment. It is useful for CI/CD pipelines, data processing, and machine learning workflows. GitHub Actions is a powerful automation tool that helps in continuous integration and deployment directly from GitHub repositories.

In this article, we will integrate Argo Workflows with GitHub Actions step-by-step

Benefits of Using Argo Workflows with GitHub Actions

  1. Scalability: Argo Workflows run natively on Kubernetes, making it highly scalable.
  2. Reusability: Workflow templates can be reused across multiple projects.
  3. Parallel Execution: Supports DAG (Directed Acyclic Graph) for executing tasks in parallel.
  4. Automation: Seamlessly integrates with GitHub Actions for automated deployments and testing.
  5. Observability: Provides a UI and CLI for monitoring and managing workflows.
  6. Cost-Effective: Utilizes Kubernetes resources efficiently, reducing infrastructure costs.

Real-Time Use Cases

  1. CI/CD Pipelines: Automate application build, test, and deployment across multiple environments.
  2. Machine Learning Pipelines: Orchestrate data preprocessing, model training, and deployment workflows.
  3. Data Processing: Process large-scale ETL (Extract, Transform, Load) jobs efficiently.
  4. Backup and Disaster Recovery: Automate periodic backups and recovery workflows for cloud-native applications.
  5. Event-Driven Workflows: Trigger workflows based on GitHub events, cloud storage changes, or API calls.

Prerequisites

Before we begin, ensure that you have the following.

  • A Kubernetes cluster (Minikube, AKS, EKS, or GKE)
  • kubectl installed and configured
  • Argo Workflows installed on your Kubernetes cluster
  • A GitHub repository with workflows enabled

Argo Workflow Setup on AKS Cluster

Step 1. Install Argo Workflows on Kubernetes.

First, install Argo Workflows using kubectl.

$ kubectl create namespace argo
$ kubectl apply -n argo -f https://github.com/argoproj/argo-workflows/releases/download/v3.4.4/install.yaml

Use the below command to check if Argo Workflows is installed correctly.

$ kubectl get pods -n argo

If you see running pods, that means the installation is successful.

Note. Here I am disabling the authentication by patching the 'argo-server' deployment to use '--auth-mode=server'. This allows access without authentication, useful for debugging in local environments.

Warning: This exposes the Argo UI without authentication, so use it cautiously in secured environments.

$ kubectl patch deployment argo-server -n argo --type='json' -p='[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value": "--auth-mode=server"}]'

$ kubectl rollout restart deployment argo-server -n argo

Step 2. Expose the Argo UI.

To access the Argo Workflows UI, run.

kubectl -n argo port-forward deployment/argo-server 8080:2746

Now, open http://localhost:8080 in your browser.

Browser

Step 3. Create a Simple Workflow.

Create a workflow definition file sample_deploy.yml in the root folder of your repository.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: github-triggered-workflow-
  namespace: argo
spec:
  entrypoint: main
  templates:
    - name: main
      container:
        image: python:3.8
        command: ["python", "-c"]
        args: ["print('Hello from Argo Workflow...')"]

Step 4. Integrate Argo Workflows with GitHub Actions.

Note. Here I am using AKS admin credentials to provide direct cluster access via kubeconfig, bypassing Azure AD authentication, and making it simple for full control and troubleshooting. but below are some best and alternate approaches for your reference.

  • Azure AD Authentication (RBAC-based): For controlled access with role-based permissions.
  • Service Principal or Managed Identity: For automated deployments with least-privilege access.
  • Static Kubeconfig with Certificates: For long-term, non-AAD authentication in isolated environments.

To Generate Base64-Encoded Kubeconfig

$ az aks get-credentials --resource-group jd-practice --name aks-argo-cluster --admin --file kubeconfig
$ cat kubeconfig | base64 -w 0

Store in GitHub Secrets

  • Go to GitHub Repository - Settings - Secrets and variables - Actions.
  • Click New repository secret and set.
  • Name: KUBE_CONFIG_DATA
  • Value: (Paste the base64 output)

Now, let's automate workflow execution using GitHub Actions.

Create a .github/workflows/argo-workflow.yml file

Add the following content.

name: Deploy to AKS without Azure AD

on:
  push:
    branches:
      - main

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout Code
        uses: actions/checkout@v3

      - name: Verify File Exists
        run: |
          if [ ! -f sample_deploy.yml ]; then
            echo "Error: sample_deploy.yml not found!"
            exit 1
          fi
          echo "Found sample_deploy.yml"

      - name: Setup Kubeconfig
        run: |
          mkdir -p $HOME/.kube
          echo "${{ secrets.KUBE_CONFIG_DATA }}" | base64 --decode > $HOME/.kube/config
          chmod 600 $HOME/.kube/config
          export KUBECONFIG=$HOME/.kube/config
          kubectl config view --minify
          kubectl get nodes

      - name: Deploy Argo Workflow
        run: |
          kubectl create -f sample_deploy.yml -n argo || true
          kubectl get workflows -n argo

Commit and push this file to your GitHub repository.

This workflow.

  • Runs on every push to the main.
  • Sets up kubectl to use AKS credentials.
  • Deploys an Argo Workflow from sample_deploy.yml.

View Deployment in GitHub Actions.

  • Go to GitHub Repository - Actions - Deploy Argo Workflow.
  • Click on the latest workflow run to see logs.
  • Ensure kubectl application runs successfully.
    GitHub Actions

Step 5. View the workflow details on Argo UI.

Find your workflow name.

Workflow name

Click on it to see logs, status, and execution details.

Execution details

Logs

Conclusion

In this article, we successfully set up and deployed a simple "Hello World" workflow using Argo Workflows on AKS. We also explored how to authenticate, deploy workflows via GitHub Actions, and view executions in the Argo UI.

This was just a basic example of getting started with Argo Workflows, you can automate complex workflows, build CI/CD pipelines, process large datasets, and integrate event-driven automation. As you move forward, you can experiment with multi-step workflows, parameterized executions, and advanced scheduling to enhance automation in your Kubernetes environment.

By leveraging Argo Workflows, you unlock powerful orchestration capabilities that simplify DevOps and data workflows in cloud-native environments.

Membership not found