A container image that works in production is not always easy to rebuild later.
The source code may still exist, but the original CI environment may have changed. Base images may have new versions, package repositories may have moved, build arguments may be different, and the exact build configuration may no longer be obvious.
This creates a practical problem for development and operations teams.
If an image was built by CI three months ago, can the team determine exactly how it was produced and reproduce that build when needed?
Docker build provenance helps answer that question.
Build provenance records information about the build process and the materials used to create an image. When enabled with Buildx, provenance can become part of the image's supply-chain metadata and provide useful information for auditing, troubleshooting, and historical build analysis.
The goal is not simply to store another piece of metadata. The goal is to make container builds easier to understand and reproduce.
What Is Docker Build Provenance?
Build provenance is metadata describing how a container artifact was built.
Depending on the configuration, it can contain information such as:
A simplified workflow looks like this:
Source Repository
|
v
CI Pipeline
|
v
Docker Buildx
|
+----------------+
| |
v v
Container Image Provenance
| |
+-------+--------+
|
v
Registry
The image is the artifact used by applications, while provenance provides information about how that artifact was created.
This distinction becomes important when investigating an older image.
Why CI Builds Need Provenance
Local builds and CI builds are not always identical.
A developer might run:
docker buildx build .
while CI runs something closer to:
docker buildx build \
--platform linux/amd64 \
--provenance=mode=max \
--cache-from=type=registry,ref=example.com/app:cache \
--cache-to=type=registry,ref=example.com/app:cache,mode=max \
--push \
-t example.com/app:$VERSION .
The CI build contains more environmental and configuration details.
If a production image later needs investigation, knowing only the image tag may not be enough.
For example:
app:1.8.2
does not immediately tell you:
Which commit?
Which Dockerfile?
Which build configuration?
Which platform?
Which build system?
Which source materials?
Provenance is intended to preserve this information with the artifact.
Enabling Provenance With Buildx
Buildx can generate provenance during an image build.
A common command is:
docker buildx build \
--provenance=mode=max \
--push \
-t example.com/myapp:1.0 \
.
The mode=max option requests a more complete provenance record than the minimal default.
For CI pipelines where reproducibility and supply-chain visibility matter, recording richer provenance can provide more useful information for later analysis.
The important part is to enable provenance during the original build.
You cannot reliably reconstruct information that was never recorded.
Minimal and Maximum Provenance
Buildx supports different levels of provenance detail.
A simplified comparison is:
Mode | Purpose | Amount of information |
|---|
min
| Basic build identification | Lower |
max
| More complete build details | Higher |
For example:
docker buildx build \
--provenance=mode=min \
--push \
-t example.com/myapp:1.0 \
.
or:
docker buildx build \
--provenance=mode=max \
--push \
-t example.com/myapp:1.0 \
.
The appropriate level depends on the organization's requirements.
If the primary objective is detailed historical analysis and reproducibility, richer provenance is generally more useful.
However, teams should understand what information is being recorded before applying a configuration across all repositories.
What Provenance Does Not Guarantee
Provenance is useful, but it does not automatically make every build bit-for-bit reproducible.
Consider:
Build A
|
v
Source commit
|
v
Base image
|
v
Package repository
If the base image or package repository resolves to different content during a later build, the result can change.
A provenance record can tell you what was used during the original build, but the required materials must still be available and reproducible.
This is why provenance should be combined with:
Immutable source references
Pinned dependencies
Immutable image digests
Reproducible build inputs
Controlled CI environments
Artifact retention
Provenance provides evidence about the build. It is not a substitute for deterministic build engineering.
Inspecting Build Provenance
After publishing an image, you may need to inspect its provenance.
A useful approach is to use Docker's build tooling to inspect image metadata and attestations.
For example:
docker buildx imagetools inspect \
example.com/myapp:1.0
The exact output depends on how the image was published and which attestations are attached.
For a multi-platform image, the result can also help identify the platforms associated with the published artifact.
The key question during an investigation is whether the image contains the expected provenance attestation.
Provenance in a CI Pipeline
A production CI pipeline can explicitly generate provenance during the image build.
For example:
steps:
- name: Build container
run: |
docker buildx build \
--platform linux/amd64 \
--provenance=mode=max \
--push \
-t registry.example.com/orders:${GIT_SHA} \
.
Using the Git commit SHA as part of the image tag creates a useful relationship between the source revision and the artifact.
The provenance then provides another layer of evidence about how the image was created.
A simplified pipeline becomes:
Git Commit
|
v
CI Build
|
+----> Image
|
+----> Provenance
|
v
Registry
This relationship is useful during incident response.
Why Git Commit References Matter
Tags can be changed.
For example:
latest
may point to a different image tomorrow.
A Git commit SHA is much more stable as a source reference.
Instead of building only:
-t example.com/myapp:latest
a CI pipeline can use:
-t example.com/myapp:${GIT_SHA}
and optionally publish a human-friendly release tag as well.
For example:
docker buildx build \
--provenance=mode=max \
--push \
-t example.com/myapp:${GIT_SHA} \
-t example.com/myapp:${VERSION} \
.
The tag provides discoverability, while the immutable digest and source revision provide stronger artifact identity.
Build Provenance and Supply Chain Security
Container supply-chain security increasingly depends on being able to answer basic questions about an artifact.
For example:
Who built it?
What source created it?
Which build system produced it?
What materials were used?
Which version of the source was involved?
Provenance helps provide those answers.
It can also be consumed by security and compliance tooling that understands software supply-chain attestations.
A typical security workflow looks like:
Container Image
|
v
Provenance Attestation
|
v
Policy / Verification
|
v
Deploy or Reject
For example, an organization could establish a policy that production images must contain provenance generated by its approved CI system.
The exact verification mechanism depends on the organization's tooling.
Reproducing an Earlier CI Build
Suppose a production image has been running for several months.
The team has:
Image digest
Source repository
Git revision
Build provenance
CI configuration
A reproduction process can then start from the recorded information.
The workflow may look like:
Production Image
|
v
Inspect Provenance
|
v
Identify Source Revision
|
v
Recover Build Configuration
|
v
Rebuild
|
v
Compare Results
The comparison does not always need to be a byte-for-byte comparison.
Depending on the objective, the team may compare:
Image digest
Image configuration
Package versions
Layer contents
Vulnerability results
Build metadata
Application behavior
For security investigations, the important question may be whether the recreated image contains the same vulnerable package rather than whether every byte is identical.
Reproducing a Build in a Controlled Environment
A reproduction environment should be as close as practical to the original CI environment.
For example:
Original CI
|
+-- BuildKit version
+-- Dockerfile
+-- Source commit
+-- Build arguments
+-- Base image
+-- Dependency versions
Reproduction:
Controlled Builder
|
+-- Matching BuildKit
+-- Same Dockerfile
+-- Same source commit
+-- Same build arguments
+-- Same pinned materials
The closer these environments are, the easier it becomes to explain differences.
If the original build used an ephemeral CI runner with one BuildKit version and the reproduction uses a different builder configuration, differences may appear even when the source is identical.
Avoid Putting Secrets Into Provenance
Provenance records build information, but it should not be treated as a secret store.
A Docker build may use:
--secret
or:
--ssh
for private credentials.
For example:
docker buildx build \
--secret id=npmrc,src=$HOME/.npmrc \
--push \
-t example.com/myapp:1.0 \
.
Credentials should remain in secure CI systems.
Do not place passwords, API keys, or private keys into Dockerfiles or build arguments simply because you want them associated with a build.
Build arguments can become visible in build metadata depending on how they are used.
For secrets, use BuildKit's secret and SSH mechanisms.
Provenance and Build Cache
Build cache and provenance solve different problems.
Cache answers:
Can this build step reuse previous work?
Provenance answers:
How was this artifact produced?
A CI pipeline can use both:
docker buildx build \
--cache-from=type=registry,ref=example.com/app:cache \
--cache-to=type=registry,ref=example.com/app:cache,mode=max \
--provenance=mode=max \
--push \
-t example.com/app:$GIT_SHA \
.
The cache improves build performance.
The provenance improves traceability.
Neither should be treated as a replacement for the other.
Common Problems
Provenance was not enabled
If an older image does not contain the required attestation, there may be insufficient information to reconstruct the original build.
This is why provenance should be enabled as part of the standard CI build process.
Source material changed
A later build may resolve different dependencies or base images.
Use immutable references where practical.
The CI environment changed
A newer BuildKit version or different build configuration can produce different results.
Record builder versions as part of the build environment documentation.
Tags were reused
A mutable tag such as latest is not sufficient for historical artifact identification.
Use immutable digests and source revisions.
Secrets were handled incorrectly
Never attempt to make reproduction easier by embedding secrets into source code or Docker build arguments.
Use secure build-time secret mechanisms.
Best Practices
Generate provenance for production images
Production artifacts should have enough metadata to support future investigation.
For example:
docker buildx build \
--provenance=mode=max \
--push \
-t example.com/myapp:${GIT_SHA} \
.
Use immutable image references
Record image digests in deployment systems.
A tag identifies an artifact logically, while a digest identifies the exact image content.
Keep CI configuration versioned
The Dockerfile alone may not describe the complete build process.
Store important Buildx, BuildKit, platform, cache, and provenance configuration alongside the application or infrastructure configuration.
Preserve source repositories
Historical artifacts are easier to reproduce when their source revisions remain accessible.
Test reproduction periodically
Do not wait for a production incident.
Select historical artifacts periodically and verify that their provenance can still be inspected and that the required source materials remain available.
Advantages and Disadvantages
Advantages | Disadvantages |
|---|
Improves visibility into how images were built | Additional metadata is generated |
Helps investigate historical artifacts | Provenance alone does not guarantee deterministic builds |
Supports supply-chain security | Older images may lack sufficient provenance |
Connects artifacts with source and build information | Reproduction still requires original materials |
Useful during security investigations | CI configuration and builder differences can affect results |
Works well with Buildx-based pipelines | Teams need policies for retention and verification |
Production Checklist
Before relying on Docker build provenance for CI reproduction, verify:
Production images generate provenance.
The chosen provenance mode contains the required information.
Source revisions are recorded.
Image digests are retained.
Base images use appropriate immutable references.
Build configuration is versioned.
BuildKit and Buildx versions are known.
Secrets are handled through secure build mechanisms.
Historical source repositories remain accessible.
CI can inspect and verify provenance from published images.
Reproduction testing is performed periodically.
Security teams know how provenance is used during artifact investigations.
Summary
Docker build provenance provides a practical record of how a container image was created.
For CI pipelines, it can connect a published image with its source revision, build configuration, materials, and build environment. This becomes especially useful when an older production image needs to be investigated or reproduced.
However, provenance should not be confused with complete reproducibility. A provenance record can describe the original build, but reproduction still depends on the availability of the recorded source materials and a sufficiently controlled build environment.
A strong container build process combines provenance with immutable image references, pinned dependencies, versioned CI configuration, controlled builders, and appropriate artifact retention.
When these pieces are used together, a production image becomes more than something that can be deployed. It becomes an artifact that can be traced back, investigated, and potentially rebuilt when the original build needs to be understood again.
Join the conversation! Your thoughts help the community grow.