Docker builds can become a bottleneck when a project contains multiple services, multiple images, or several independent build targets.
A common reaction is to increase build concurrency.
That can help, but more parallelism does not automatically mean faster builds.
Docker Buildx already uses BuildKit, which can execute independent build steps in parallel. Buildx Bake can also run multiple targets concurrently. These are related, but they control different levels of concurrency.
Understanding that difference is important before changing CI configuration or increasing the number of concurrent build jobs.
Too little concurrency can leave CPU and network resources unused. Too much concurrency can cause memory pressure, registry throttling, disk contention, or slower builds.
This article explains where Buildx parallelism comes from, when increasing it helps, and how to configure it safely.
How Docker Build Parallelism Works
There are two different levels to understand.
The first is parallelism between Buildx Bake targets.
For example, a project might contain:
frontend
backend
worker
migration
These can be independent container images.
The second level is parallelism inside an individual BuildKit build.
BuildKit represents the build as a dependency graph and can execute independent operations concurrently.
A simplified model looks like this:
Build
|
+---------+---------+
| |
frontend backend
| |
npm install dotnet restore
| |
build build
If two operations do not depend on each other, BuildKit can potentially execute them at the same time.
Increasing Bake concurrency does not necessarily increase this internal solver parallelism.
Buildx Bake and Parallel Targets
Buildx Bake is useful when a repository contains multiple build targets.
A simple docker-bake.hcl file might contain:
group "default" {
targets = ["frontend", "backend", "worker"]
}
target "frontend" {
context = "./frontend"
dockerfile = "Dockerfile"
tags = ["example/frontend:latest"]
}
target "backend" {
context = "./backend"
dockerfile = "Dockerfile"
tags = ["example/backend:latest"]
}
target "worker" {
context = "./worker"
dockerfile = "Dockerfile"
tags = ["example/worker:latest"]
}
Running:
docker buildx bake
allows the targets in the group to run concurrently.
Docker documents Bake as a high-level build command where specified targets run in parallel.
This is often more useful than writing several independent docker buildx build commands in a CI script.
Limiting Bake Concurrency
Running every target at the same time is not always appropriate.
Suppose a CI runner has limited CPU and memory.
You could limit Bake to two targets:
docker buildx bake --jobs=2
This is equivalent to:
docker buildx bake --jobs=parallel=2
The important point is that this controls target concurrency.
It does not set the number of BuildKit operations allowed inside each target. Docker explicitly distinguishes Bake target concurrency from BuildKit's internal build-step parallelism.
For example:
Bake
|
+-- frontend target ----> BuildKit solver
|
+-- backend target -----> BuildKit solver
With:
--jobs=2
at most two Bake targets are scheduled concurrently.
Each target can still have multiple independent BuildKit operations running inside it.
When Increasing Build Concurrency Helps
Increasing concurrency usually helps when there is enough work that can safely run at the same time.
Consider three independent services:
frontend: 2 minutes
backend: 3 minutes
worker: 2 minutes
If built sequentially, the approximate execution time could approach:
2 + 3 + 2 = 7 minutes
If the three builds can run concurrently and the machine has sufficient resources, total wall-clock time can move closer to the duration of the slowest target.
frontend |----2 min----|
backend |-------3 min-------|
worker |----2 min----|
The actual result depends on CPU, memory, network bandwidth, cache behavior, registry performance, and dependencies between targets.
Parallelism reduces waiting only when the system has capacity to perform the additional work.
When More Parallelism Makes Builds Slower
The most common mistake is assuming that a higher concurrency value is always better.
Imagine a CI runner with:
4 CPU cores
8 GB RAM
Running four large builds simultaneously may create resource contention.
Each build could be compiling code, extracting image layers, running package managers, and writing cache data.
Instead of:
Build A -> fast
Build B -> fast
Build C -> fast
Build D -> fast
you may get:
Build A -> CPU contention
Build B -> memory pressure
Build C -> disk contention
Build D -> network contention
The result can be longer overall build time.
This is why concurrency should be measured rather than selected arbitrarily.
Configure BuildKit Solver Parallelism
BuildKit provides a separate control for limiting the number of parallel build steps.
A BuildKit configuration can contain:
[worker.oci]
max-parallelism = 4
This controls the maximum number of parallel build steps for the OCI worker.
Docker recommends this type of configuration particularly for low-powered machines.
You can use the configuration when creating a docker-container builder:
docker buildx create \
--use \
--name mybuilder \
--driver docker-container \
--buildkitd-config /etc/buildkitd.toml
Then inspect the builder:
docker buildx inspect --bootstrap
This gives you a separate control from Bake's --jobs setting.
Bake Concurrency vs BuildKit Parallelism
The difference is easier to understand in a table.
Setting | Controls | Example |
|---|
Bake --jobs | Number of Bake targets running concurrently | --jobs=2
|
BuildKit max-parallelism | Parallel build steps inside the worker | max-parallelism = 4
|
CPU resources | Available compute capacity | 4 CPU cores |
Memory | Available memory for concurrent operations | 8 GB |
Registry concurrency | Registry connections used during builds | BuildKit configuration |
These settings should not be treated as interchangeable.
For example, setting:
docker buildx bake --jobs=4
does not mean that the entire BuildKit system is limited to four operations.
It means four Bake targets can be scheduled concurrently.
A Practical CI Configuration
Consider a repository containing:
services/
api/
web/
worker/
reporting/
A Bake file can define each image:
group "default" {
targets = [
"api",
"web",
"worker",
"reporting"
]
}
target "api" {
context = "./services/api"
dockerfile = "Dockerfile"
}
target "web" {
context = "./services/web"
dockerfile = "Dockerfile"
}
target "worker" {
context = "./services/worker"
dockerfile = "Dockerfile"
}
target "reporting" {
context = "./services/reporting"
dockerfile = "Dockerfile"
}
Instead of immediately running all four targets:
docker buildx bake
the CI pipeline could start with:
docker buildx bake --jobs=2
Then compare the build duration and runner resource usage.
If the runner consistently has spare CPU and memory, increasing the value may be reasonable.
If memory usage reaches the runner limit, reducing concurrency may produce a more stable build.
Multi-Platform Builds Need Additional Consideration
Parallelism becomes more complicated when building for multiple platforms.
For example:
docker buildx build \
--platform linux/amd64,linux/arm64 \
--push \
-t example.com/myapp:latest .
Buildx can build multiple platforms and combine the results into a manifest list.
However, multi-platform builds can consume substantially more resources, especially when emulation is involved.
Before increasing concurrency, determine whether the builder has native support for the target platforms.
You can inspect supported platforms with:
docker buildx inspect --bootstrap
If multiple large targets are already building for multiple architectures, aggressive concurrency can quickly overwhelm a CI runner.
Resource Limits Inside Build Steps
Buildx also supports resource controls for build containers.
For example:
docker buildx build \
--resource memory=2g \
--resource cpu-quota=50000 \
--resource cpu-period=100000 \
.
These limits apply to individual build steps rather than the entire build. Docker notes that because BuildKit can execute build steps in parallel, the limits are applied per step.
This distinction matters.
If five build steps are active and each can use up to a specified resource limit, the total resource demand can be substantially higher than the limit assigned to one step.
Therefore, per-step resource limits should be considered together with overall concurrency.
Caching Changes the Concurrency Equation
Build caching can change whether additional concurrency is useful.
Suppose a target has a warm cache:
Source
|
v
Cached dependency layer
|
v
Cached build layer
|
v
Image
The target may finish quickly.
Another target may have an invalidated dependency layer and require a full rebuild.
In that situation, increasing target concurrency may not provide a proportional reduction in total build time.
Use cache effectively with options such as:
docker buildx build \
--cache-from=type=registry,ref=example.com/myapp:buildcache \
--cache-to=type=registry,ref=example.com/myapp:buildcache,mode=max \
.
The exact cache strategy should match the CI environment.
The goal is not simply to run more builds. It is to reduce the amount of work each build needs to perform.
Registry and Network Bottlenecks
Container builds frequently depend on external registries.
Multiple concurrent builds may simultaneously pull:
base images
package layers
dependency artifacts
cache layers
They may also push multiple image layers after completion.
BuildKit has registry connection limits to avoid excessive concurrent connections to a registry. Docker documents a default limit of four simultaneous connections per registry for pulling and pushing, plus an additional metadata connection.
Therefore, increasing application-level build concurrency may eventually move the bottleneck from CPU to network or registry throughput.
This is another reason to monitor the complete pipeline instead of measuring only the Docker command duration.
Common Mistakes
Setting concurrency based on CPU count alone
CPU is only one resource.
Memory, disk I/O, network bandwidth, registry performance, and cache storage also matter.
Confusing Bake concurrency with BuildKit parallelism
These controls operate at different levels.
--jobs controls Bake targets.
max-parallelism controls BuildKit worker execution.
Running every target concurrently
Small projects may tolerate this, but larger monorepos can contain dozens of targets.
Unrestricted concurrency can overload shared CI runners.
Ignoring multi-platform builds
Building amd64 and arm64 can increase resource requirements significantly.
Test concurrency using the same platform matrix used by production releases.
Measuring only successful builds
A concurrency setting that is fast during normal builds may become problematic when caches are cold.
Benchmark both warm-cache and cold-cache scenarios.
Best Practices
Start with conservative concurrency
Use a small value such as:
docker buildx bake --jobs=2
and measure the result.
Increase it only when the CI runner has enough capacity.
Separate independent targets
Bake works best when the build targets have clear boundaries.
Avoid unnecessary dependencies that force targets to wait for each other.
Monitor resource utilization
Track:
CPU utilization
Memory usage
Disk I/O
Network throughput
Registry latency
Build duration
Cache hit rate
A faster build command is not useful if it causes CI runners to become unstable.
Tune concurrency per runner type
A developer laptop and a large CI runner should not necessarily use the same configuration.
For example:
Developer machine -> 1–2 concurrent targets
Standard CI runner -> 2–4 targets
Large CI runner -> higher concurrency after testing
These are starting points, not universal limits.
Keep BuildKit configuration explicit
If your team depends on a specific solver limit, store the BuildKit configuration with the infrastructure or CI configuration.
For example:
[worker.oci]
max-parallelism = 4
This makes the build environment easier to reproduce.
Advantages and Disadvantages
Advantages | Disadvantages |
|---|
Can reduce wall-clock build time | Higher resource consumption |
Useful for monorepos with multiple images | Too much concurrency can slow builds |
Works well with Buildx Bake | Requires CI resource monitoring |
Can improve utilization of capable builders | Registry and network limits can become bottlenecks |
BuildKit already parallelizes independent steps | Parallelism does not help dependent steps |
Concurrency can be explicitly controlled | Different runners may need different settings |
When Should You Increase Build Concurrency?
Increase concurrency when:
Multiple independent targets exist.
The CI runner has unused CPU and memory.
Registry throughput is sufficient.
Build cache performance is good.
Current builds spend significant time waiting for independent targets.
Build duration has been measured under realistic conditions.
Do not increase concurrency simply because a build is slow.
First identify where the time is being spent.
If one Dockerfile contains a long sequential compilation step, adding more Bake targets will not make that individual step faster.
If several independent images are waiting to build, however, parallel target execution can provide a meaningful improvement.
Production Checklist
Before increasing Buildx build concurrency, check:
How many independent build targets exist?
Are those targets already being built concurrently?
Is the bottleneck CPU, memory, disk, network, or registry access?
What does docker buildx inspect --bootstrap report?
Is the builder using a custom BuildKit configuration?
What is the current max-parallelism value?
Are CI runners using warm or cold caches?
Are multiple platforms being built?
Does the registry handle the additional traffic?
Does the higher concurrency improve total pipeline time without increasing failure rates?
Summary
Docker Buildx already provides parallel execution at multiple levels.
BuildKit can execute independent steps in a build graph concurrently, while Buildx Bake can run multiple build targets at the same time. These mechanisms solve different problems.
The --jobs option on docker buildx bake controls how many Bake targets run concurrently. BuildKit's max-parallelism setting controls how many build steps a worker can execute in parallel.
The right configuration depends on the available CPU, memory, disk, network, registry performance, cache behavior, and platform matrix.
For most teams, the safest approach is to start with conservative concurrency, measure real CI builds, and increase the limit gradually. The goal should be shorter and more predictable builds, not the highest possible concurrency number.
Join the conversation! Your thoughts help the community grow.