Distributed AI and machine learning workloads often need several Pods to run together. A training job may require multiple workers, and starting only part of those workers can leave the workload waiting indefinitely while consuming cluster resources.

Traditional Kubernetes scheduling evaluates Pods individually. That works well for many applications, but it is not always suitable for tightly coupled workloads.

Kubernetes 1.37 graduates gang scheduling to Beta, expanding native workload-aware scheduling through the Workload API and PodGroup concept. Gang scheduling uses an all-or-nothing approach so that a defined group of Pods is scheduled only when the cluster can accommodate the required group. Kubernetes 1.37 also introduces workload-aware preemption and PodGroup queueing improvements as part of this scheduling model.

For distributed AI workloads, this can help avoid a common problem: partially scheduled workers consuming resources without allowing the training job to make useful progress.

What Is Gang Scheduling?

Gang scheduling treats a group of Pods as a single scheduling unit.

With normal scheduling:

Pod 1 → Scheduled
Pod 2 → Scheduled
Pod 3 → Pending
Pod 4 → Pending

The workload has started partially, but it may not be able to perform useful work.

With gang scheduling:

Pod 1 ┐
Pod 2 │
Pod 3 ├──> Schedule together
Pod 4 ┘

If the cluster cannot satisfy the group's scheduling requirement, the scheduler does not bind the required Pods.

The Kubernetes documentation describes gang scheduling as an all-or-nothing policy controlled by a minCount value.

Why Distributed AI Workloads Need It

Consider a distributed training workload with four workers:

Training Job
    |
    +-- Worker 1
    +-- Worker 2
    +-- Worker 3
    +-- Worker 4

The training framework may expect all four workers to participate.

If Kubernetes schedules only two:

Worker 1 → Running
Worker 2 → Running
Worker 3 → Pending
Worker 4 → Pending

the running workers may wait for the remaining workers.

This can result in:

Kubernetes identifies distributed AI/ML and HPC workloads as important use cases for gang scheduling because partial scheduling can prevent a workload from making progress.

PodGroup: The Scheduling Unit

The PodGroup API represents a group of Pods that should be scheduled together.

A simplified PodGroup can define:

apiVersion: scheduling.k8s.io/v1beta1
kind: PodGroup
metadata:
  name: training-workers
spec:
  schedulingPolicy:
    gang:
      minCount: 4

The important setting is:

minCount: 4

This tells the scheduler that at least four Pods must be schedulable together for the gang to be admitted.

The exact API version and feature-gate configuration should be verified against the Kubernetes 1.37 cluster configuration because workload-aware scheduling capabilities have been evolving across releases.

How Gang Scheduling Works

The scheduling process can be simplified into four stages:

1. Workload submitted
        ↓
2. Pods associated with PodGroup
        ↓
3. Scheduler evaluates group
        ↓
4. Group scheduled or remains pending

When the scheduler evaluates a PodGroup, it considers the group as a scheduling unit rather than treating every Pod as an unrelated request.

If the group cannot satisfy its scheduling requirements:

Required: 4 Pods
Available capacity: 3 Pods

Result:
No gang admission

If enough resources become available:

Required: 4 Pods
Available capacity: 4+ Pods

Result:
Gang can be scheduled

The PodGroup scheduling cycle uses a consistent cluster snapshot while evaluating the group and makes the scheduling decision for the group together.

Gang Scheduling With Kubernetes Jobs

Kubernetes also provides integration between Jobs and workload-aware scheduling.

The Job controller can translate a Job's scheduling configuration into Workload and PodGroup objects when the appropriate feature gate is enabled. Kubernetes documentation describes basic and gang scheduling policies for Jobs.

A Job can therefore express scheduling intent instead of requiring developers to manually construct every workload object.

Conceptually:

apiVersion: batch/v1
kind: Job
metadata:
  name: distributed-training
spec:
  parallelism: 4
  completions: 4
  scheduling:
    schedulingPolicy: gang

The exact manifest should be validated against the Kubernetes 1.37 API and enabled feature gates before production use because workload-aware Job integration remains dependent on the relevant feature configuration.

Gang Scheduling for .NET Workloads

Although gang scheduling is particularly relevant to AI/ML workloads, the same scheduling model can be useful for distributed .NET batch processing.

For example, imagine a data-processing job that requires four workers:

.NET Batch Job
      |
      +-- Worker 1
      +-- Worker 2
      +-- Worker 3
      +-- Worker 4

Each worker might execute a different partition of a large dataset.

A simplified .NET worker could process its assigned partition:

public sealed class DataProcessor
{
    public async Task ProcessAsync(
        int partitionId,
        CancellationToken cancellationToken)
    {
        var records =
            await LoadPartitionAsync(
                partitionId,
                cancellationToken);

        foreach (var record in records)
        {
            await ProcessRecordAsync(
                record,
                cancellationToken);
        }
    }

    private Task<IReadOnlyList<DataRecord>> LoadPartitionAsync(
        int partitionId,
        CancellationToken cancellationToken)
    {
        // Load the assigned partition.
        return Task.FromResult<IReadOnlyList<DataRecord>>(
            Array.Empty<DataRecord>());
    }

    private Task ProcessRecordAsync(
        DataRecord record,
        CancellationToken cancellationToken)
    {
        // Process the record.
        return Task.CompletedTask;
    }
}

public record DataRecord(long Id);

If the processing framework expects all workers to be available before beginning, gang scheduling can prevent Kubernetes from starting only part of the worker group.

Gang Scheduling vs Normal Scheduling

AreaStandard SchedulingGang Scheduling
Scheduling unitIndividual PodPodGroup
Partial schedulingAllowedControlled by minCount
Distributed trainingCan be inefficientBetter suited
Resource fragmentationPossibleReduced for grouped workloads
Simple web applicationsExcellent fitUsually unnecessary
Batch workloadsDepends on designUseful when workers are tightly coupled
Scheduling complexityLowerHigher

Gang scheduling is not a replacement for standard Kubernetes scheduling.

For a typical ASP.NET Core Deployment:

3 replicas

there is usually no requirement for all three Pods to start simultaneously.

For a distributed training job:

4 workers required

the scheduling requirement is fundamentally different.

Preventing Resource Deadlocks

One of the main problems gang scheduling addresses is a scheduling deadlock.

Imagine two workloads:

Workload A → Requires 4 GPUs
Workload B → Requires 4 GPUs

The cluster has eight GPUs, but the resources are fragmented across nodes.

Without workload-aware scheduling, individual Pods can consume available capacity:

A1 → Scheduled
A2 → Scheduled
B1 → Scheduled
B2 → Scheduled

Now neither workload has enough resources to start completely.

The workloads may remain stuck while resources are technically being consumed.

Gang scheduling changes the admission decision so the scheduler considers the group rather than blindly placing individual Pods. Kubernetes specifically identifies this partial-scheduling problem as a source of deadlocks and inefficient resource utilization.

Workload-Aware Preemption

Kubernetes 1.37 also advances workload-aware preemption.

Traditional preemption generally considers individual Pods. Workload-aware preemption can reason about PodGroups as scheduling units.

Conceptually:

High-priority training job
        ↓
Needs additional capacity
        ↓
Scheduler evaluates workload
        ↓
Selects appropriate victims
        ↓
Creates capacity for the workload

The Kubernetes documentation describes workload-aware preemption as evaluating the cluster as a whole and considering PodGroup priority and disruption behavior when selecting workloads to preempt.

This matters for AI clusters where a large distributed workload may require capacity across multiple nodes rather than on a single node.

Topology-Aware Scheduling

Distributed AI workloads can also benefit from topology-aware placement.

For example:

Cluster
  |
  +-- Zone A
  |     +-- GPU Node
  |     +-- GPU Node
  |
  +-- Zone B
        +-- GPU Node
        +-- GPU Node

If workers communicate heavily, spreading them across distant infrastructure can increase communication overhead.

Topology-aware workload scheduling allows scheduling policies to consider topology domains such as zones or racks. Kubernetes documentation specifically describes its use with gang scheduling for tightly coupled AI/ML workloads.

This produces a more complete scheduling strategy:

Gang Scheduling
      +
Topology Awareness
      +
Resource Availability
      ↓
Better workload placement

Checking PodGroup Status

When troubleshooting workload-aware scheduling, inspect the PodGroup:

kubectl get podgroups

For detailed information:

kubectl get podgroup training-workers -o yaml

You can also inspect scheduling conditions.

A successfully scheduled PodGroup can report a PodGroupScheduled condition. An unschedulable group can expose reasons such as insufficient resources or invalid scheduling constraints.

This is useful because:

Pod Pending

is less informative than:

PodGroup unschedulable
Reason: insufficient capacity

The latter gives the platform team a clearer indication that the problem is at the workload scheduling level.

Common Mistakes

Using Gang Scheduling for Every Application

A normal ASP.NET Core service usually does not need all replicas scheduled simultaneously.

Gang scheduling should be reserved for workloads with a genuine group-level scheduling requirement.

Setting minCount Too High

If a cluster regularly cannot provide the requested number of Pods, the workload may remain pending.

For example:

minCount: 16

requires substantially more capacity than:

minCount: 4

The value should reflect the application's actual execution requirements.

Ignoring Node Capacity

Gang scheduling cannot create resources that do not exist.

If a workload requires eight GPU Pods but the cluster has capacity for only six, the workload will remain unschedulable until capacity changes.

Forgetting Topology

For tightly coupled distributed workloads, simply scheduling all Pods is not always enough. Placement across zones or other topology domains can also affect communication behavior.

Assuming PodGroupScheduled Means the Job Is Healthy

The PodGroupScheduled condition represents the initial scheduling decision. It does not automatically mean that every Pod will continue running successfully after scheduling.

Troubleshooting Gang Scheduling

When a distributed workload remains pending, use a structured process.

Step 1: Inspect the Job

kubectl describe job distributed-training

Step 2: Inspect Pods

kubectl get pods -l job-name=distributed-training

Step 3: Inspect the PodGroup

kubectl get podgroups

Then:

kubectl get podgroup distributed-training -o yaml

Step 4: Check Events

kubectl get events --sort-by=.lastTimestamp

Look for:

Step 5: Verify Cluster Capacity

Check the nodes:

kubectl get nodes

For resource details:

kubectl describe node <node-name>

The objective is to determine whether the group is genuinely unschedulable or whether the scheduling configuration is preventing placement.

Best Practices

  1. Use gang scheduling only for workloads that genuinely require group-level admission.

  2. Set minCount according to the minimum number of workers required for useful progress.

  3. Test workloads against realistic cluster capacity.

  4. Consider GPU and other extended resources explicitly.

  5. Use topology-aware scheduling when worker communication makes placement important.

  6. Monitor PodGroup conditions and scheduler events.

  7. Avoid unnecessarily large gang sizes.

  8. Test workload-aware preemption carefully in shared clusters.

  9. Keep ordinary stateless .NET services on standard scheduling unless they have a real group dependency.

  10. Document the resource requirements of distributed workloads so cluster administrators can plan capacity correctly.

Advantages and Disadvantages

Advantages

Disadvantages

Conclusion

Kubernetes 1.37's graduation of gang scheduling to Beta is an important step for workload-aware scheduling. Instead of treating every Pod as an independent scheduling request, Kubernetes can reason about tightly coupled groups and require enough capacity for the workload to make progress.

For distributed AI workloads, the difference is significant:

Traditional scheduling
Pod → Pod → Pod → Pod

Gang scheduling
      Workload
         |
    +----+----+
    |    |    |
   Pod  Pod  Pod ...
         |
   Schedule as group

The feature is not intended to replace normal Kubernetes scheduling. A typical ASP.NET Core application with independent replicas generally benefits from standard scheduling.

For distributed training, GPU workloads, HPC simulations, and tightly coupled .NET batch processing, however, group-level scheduling can provide a much better fit.

The practical approach is to define the minimum worker count carefully, verify cluster capacity, monitor PodGroup status, and combine gang scheduling with topology and preemption policies when the workload requires them.