Kubernetes 1.37 Workload-Aware Scheduling for AI and Batch Workloads

Kubernetes normally schedules Pods individually. That works well for many applications, but it can create problems when several Pods must run together before a workload can make progress.

AI training jobs and large batch workloads are common examples. A training job may need several workers to start at the same time. If Kubernetes schedules only some of those workers, the remaining Pods can stay pending while the already-running Pods consume cluster resources.

Kubernetes 1.37 moves workload-aware scheduling further by graduating the Workload and PodGroup APIs, gang scheduling, workload-aware preemption, and DRA ResourceClaims for workloads to Beta. It also adds the Alpha CompositePodGroup API for more complex hierarchical workloads.

Why Pod-by-Pod Scheduling Can Be a Problem

Consider a distributed training job that needs four worker Pods.

With normal scheduling, Kubernetes can place them one at a time:

Worker 1  -> Running
Worker 2  -> Running
Worker 3  -> Running
Worker 4  -> Pending

The first three workers may consume GPUs, CPU, memory, or other resources while the fourth worker waits.

The application may not be able to start useful work until all four workers are available.

This creates two problems:

Resources are occupied by a workload that cannot make progress.
Other workloads may be prevented from using those resources.

Gang scheduling addresses this by treating the group as a scheduling unit.

What Is Workload-Aware Scheduling?

Workload-Aware Scheduling, or WAS, adds scheduling information at the workload level.

Instead of describing only individual Pods, Kubernetes can represent a group of Pods and define how that group should be scheduled.

The main objects are:

Workload
PodGroup
CompositePodGroup

The Workload API describes the scheduling requirements and structure of a multi-Pod application.

The PodGroup represents the runtime scheduling state for a group of Pods.

In Kubernetes 1.37, these APIs are part of the Beta workload scheduling model. The Workload API uses the scheduling.k8s.io/v1beta1 API version.

A simplified relationship looks like this:

Workload
   |
   | defines scheduling intent
   v
PodGroup
   |
   | represents runtime group
   v
Pods

This separation is useful because workload controllers can manage application lifecycle while the scheduler focuses on how the related Pods should be placed.

Gang Scheduling

Gang scheduling is one of the most important parts of the Kubernetes 1.37 update for AI and batch workloads.

The basic rule is:

Schedule enough Pods for the workload to make progress, instead of treating every Pod as an independent scheduling decision.

For example:

apiVersion: scheduling.k8s.io/v1beta1
kind: Workload
metadata:
  name: training-workload
spec:
  podGroupTemplates:
    - name: workers
      schedulingPolicy:
        gang:
          minCount: 4

Here, minCount: 4 means that the scheduler should not consider the group successfully scheduled until at least four Pods can be scheduled.

The exact Pod creation and controller relationship depends on the workload controller managing the application.

This is different from simply giving four Pods the same label. A label groups objects logically, but it does not tell the scheduler that those Pods have an all-or-nothing scheduling requirement.

Why `minCount` Matters

The minCount value lets you define how many Pods need to be scheduled for the workload to make progress.

Suppose a batch job creates eight Pods:

parallelism = 8

You might not need all eight to start before useful work can begin.

You could define:

schedulingPolicy:
  gang:
    minCount: 4

The scheduler can then work toward getting four Pods placed together.

For a tightly coupled distributed training job, you may instead require all workers:

schedulingPolicy:
  gang:
    minCount: 8

The right value depends on the application.

Do not automatically set minCount equal to the total number of Pods. Some workloads can make progress with fewer workers, while others cannot.

Workload-Aware Preemption

Kubernetes 1.37 also graduates workload-aware preemption to Beta.

Traditional preemption considers Pods individually. Workload-aware preemption changes that model for PodGroups.

When a PodGroup cannot be scheduled, the scheduler can treat the group as the preemption unit instead of making independent decisions for individual Pods.

Imagine a training workload that needs four GPUs:

Node A: 1 GPU available
Node B: 1 GPU available
Node C: 2 GPUs available

The workload needs four GPUs in a placement that satisfies its scheduling requirements.

Removing one unrelated Pod from Node A might not help if the remaining placement constraints still prevent the complete workload from running.

Workload-aware preemption evaluates whether preemption can actually make the workload schedulable.

That avoids some cases where Kubernetes evicts a lower-priority Pod but the waiting workload still cannot start.

Topology-Aware Scheduling

Resource quantity is not always enough for AI workloads.

Two Nodes may both have GPUs, but the distance between those GPUs and the rest of the workload can affect communication performance.

Kubernetes already has topology-aware workload scheduling capabilities, and Kubernetes 1.37 continues to develop this area.

A PodGroup can specify a topology key:

spec:
  schedulingPolicy:
    gang:
      minCount: 4

  schedulingConstraints:
    topology:
      - key: topology.kubernetes.io/rack

This tells the scheduler that the workload has a topology requirement around the specified domain.

For distributed workloads, this can help keep related Pods within an appropriate physical or logical boundary.

Topology requirements should be based on the actual architecture of the workload. Adding unnecessary constraints can make scheduling harder because the scheduler has fewer valid placements.

DRA and Specialized Resources

AI workloads frequently need resources such as GPUs and other specialized devices.

Dynamic Resource Allocation, or DRA, provides a Kubernetes-native way to request such resources.

Kubernetes 1.37 allows DRA ResourceClaim and ResourceClaimTemplate information to be associated with workloads and PodGroups. This is useful when multiple Pods need coordinated access to specialized resources.

The model can be thought of as:

Workload
   |
   +-- PodGroup
   |
   +-- ResourceClaim
           |
           v
      Specialized device

This becomes particularly useful when scheduling depends on both the number of Pods and the resources available to those Pods.

Kubernetes 1.37 also graduates DRA extended resource support to Stable. That allows a DRA driver to satisfy traditional extended resource requests through a DeviceClass.

Workload Scheduling for Batch Jobs

Kubernetes 1.37 adds scheduling configuration directly to the Job API.

A Job can use a spec.scheduling field to configure scheduling policies, topology constraints, disruption behavior, and resource claims.

If the field is omitted, the Job keeps Basic scheduling behavior.

For a workload that needs gang scheduling, the scheduling configuration can explicitly request it. Kubernetes then uses the shared workload-building mechanism to create the corresponding scheduling resources.

This is important because application developers do not necessarily need to create separate scheduling objects manually for every Job.

The Job controller can translate the scheduling configuration into the resources used by the scheduler.

Basic Scheduling vs Gang Scheduling

The difference becomes clearer with a simple comparison.

Feature	Basic scheduling	Gang scheduling
Scheduling unit	Individual Pods	PodGroup
Partial placement	Allowed	Controlled by `minCount`
Suitable for	Independent workloads	Tightly coupled workloads
Preemption	Pod-oriented	Workload-aware
AI training	Can cause partial placement	Better fit for coordinated workers
Batch processing	Good for independent tasks	Useful when several Pods must start together

Gang scheduling is not automatically better.

If a Job consists of completely independent tasks, forcing all Pods to start together can reduce cluster utilization.

Use it when the workload actually has a group-level scheduling requirement.

CompositePodGroup for Complex Workloads

Some modern AI workloads are more complicated than a single flat group of identical Pods.

For example, a workload might contain:

Training workload
|
+-- Coordinator
|
+-- Worker group A
|     +-- Worker
|     +-- Worker
|
+-- Worker group B
      +-- Worker
      +-- Worker

Different groups may have different resource and topology requirements.

Kubernetes 1.37 introduces the Alpha CompositePodGroup API for this type of hierarchical scheduling.

It can represent multiple levels of workload structure and support multi-level topology constraints, gang scheduling, and workload-aware preemption.

Because this capability is Alpha, it should be evaluated carefully before using it for production-critical workloads.

Feature Status in Kubernetes 1.37

The maturity of each feature matters when planning adoption.

Feature	Kubernetes 1.37 status
Workload API	Beta
PodGroup API	Beta
Gang scheduling	Beta
Workload-aware preemption	Beta
DRA ResourceClaims for workloads	Beta
Scheduling building block APIs	Beta
Job scheduling integration	Alpha
CompositePodGroup	Alpha
Multi-level topology-aware scheduling	Alpha
DRA extended resource support	Stable

The Beta features are disabled by default and require the GenericWorkload feature gate where applicable. The Alpha features have their own requirements and should be tested separately.

Enabling Workload-Aware Scheduling

Before using workload-aware scheduling, verify that the required API and feature gate are available in the cluster.

The main prerequisite in Kubernetes 1.37 is the GenericWorkload feature gate for the relevant components.

For example, cluster administrators should verify the API resources before deploying workload-aware scheduling:

kubectl api-resources | grep -E 'workload|podgroup'

You can also inspect the cluster version:

kubectl version

Do not enable every scheduling feature simply because it is available.

Start with the specific capability required by the workload, test it in a non-production cluster, and then evaluate its behavior under realistic resource pressure.

Common Mistakes

Using Gang Scheduling for Every Job

Gang scheduling is designed for workloads that need coordinated placement.

Independent batch tasks often work better with normal scheduling.

Setting `minCount` Too High

A workload requiring eight Pods with:

minCount: 8

cannot make progress until all eight can be scheduled according to its constraints.

If the application can operate with four workers, setting minCount to eight may unnecessarily increase scheduling pressure.

Ignoring Topology

A distributed workload can technically run across the cluster while still performing poorly because its Pods are placed far apart.

If network locality matters, define topology requirements deliberately.

Treating Beta Features as Final

Beta features are much more mature than Alpha features, but they are still subject to evolution.

Test upgrades and review the feature status before moving a scheduling design into a long-lived production environment.

Forgetting Resource Availability

Gang scheduling does not create resources.

If the cluster does not have enough GPUs, CPU, memory, or other required resources, the workload still cannot run.

A Practical Adoption Strategy

For an AI or batch platform, introduce workload-aware scheduling gradually.

Step 1: Identify Workloads That Actually Need It

Start with workloads where partial scheduling creates a real problem.

Good candidates include:

Distributed AI training
MPI-style jobs
Large batch workloads with coordinated workers
Workloads with strict topology requirements

Step 2: Define the Minimum Useful Group

Determine how many Pods need to be available before the workload can make progress.

Use that value for minCount.

Step 3: Add Topology Only When Needed

If GPU or network locality matters, add the appropriate topology constraint.

Do not add constraints simply to make the YAML more complicated.

Step 4: Test Under Resource Pressure

A scheduling feature should be tested when the cluster is busy.

Create competing workloads and verify:

How long the workload waits
Whether partial scheduling occurs
Whether preemption helps
Whether lower-priority workloads are disrupted
Whether the workload eventually gets a valid placement

Step 5: Monitor Scheduler Behavior

Watch pending Pods, scheduling events, resource utilization, and preemption activity.

A workload-aware policy should improve the workload's ability to make progress without causing unnecessary disruption to unrelated applications.

Final Takeaway

Kubernetes 1.37 makes workload-aware scheduling much more practical for AI and batch workloads.

The Workload and PodGroup APIs provide the foundation for group-level scheduling. Gang scheduling prevents tightly coupled workloads from being treated as unrelated Pods, while workload-aware preemption considers the workload as a whole when preemption is required. DRA integration adds specialized resource allocation, and topology-aware scheduling provides more control over where related Pods are placed.

The key is to use these capabilities where the application actually needs coordinated scheduling.

For a simple batch Job, standard Pod scheduling may still be the right choice. For a distributed training job that needs several workers, specialized resources, and controlled placement, workload-aware scheduling gives Kubernetes much more information about what the application actually needs.