Keeping Git Repositories Responsive When Thousands of Agents Push Changes

Saurav Kumar

Git was designed around distributed collaboration, where developers work on local branches, commit changes, and exchange updates through a central repository or remote service. This model continues to work well for most software teams.

AI coding agents are changing the volume and frequency of Git operations. Instead of committing a few times during a development session, an agent may create checkpoints after individual tasks, push several branches, run automated tests, and fetch updated commits repeatedly. When hundreds or thousands of agents operate in the same repository, these operations can create sustained pressure on Git infrastructure.

The problem is not simply the number of commits. Every push involves object transfer, reference updates, storage operations, and potentially downstream CI activity. Every fetch may require Git to prepare and transfer packfiles. When many operations happen simultaneously, repository storage, network bandwidth, CPU, and reference coordination can become bottlenecks.

Keeping a repository responsive under this workload requires changes to both the Git workflow and the infrastructure serving it. This article examines the main bottlenecks and practical approaches for handling high-volume concurrent Git operations.

Understand Why Thousands of Agents Change Git Workloads

Traditional development usually produces an uneven workload. Developers edit code, run tests, and push changes when a task is ready.

Agents often follow shorter execution cycles:

Retrieve the latest repository state.
Modify a set of files.
Commit a checkpoint.
Push the branch.
Trigger automated validation.
Fetch updates and continue working.

When thousands of agents follow this pattern, repository activity becomes continuous rather than occasional.

There are four major sources of pressure.

Concurrent writes

Each push transfers Git objects and requests updates to one or more references. A repository receiving many pushes must process these writes while maintaining correct branch state.

Read amplification

One push can trigger several downstream reads. CI pipelines, code scanning, test environments, and other agents may all fetch the same commit.

A single branch update can therefore generate many more fetch operations than push operations.

Reference contention

When agents create independent branches, their writes can often be handled separately. However, when completed changes are merged into a shared branch such as main, those updates compete for the same reference.

Repository maintenance

Git needs to organize stored objects, remove unreachable data when appropriate, and generate efficient packfiles. As the repository grows and receives frequent updates, housekeeping becomes more important.

These workloads need to be considered together. Optimizing push latency while allowing fetch traffic or repository maintenance to overwhelm storage will not solve the overall problem.

Understand the Difference Between Git Objects and References

A Git repository contains two important categories of data: objects and references.

Objects store the repository's content and history. They include:

Blob objects containing file content.
Tree objects describing directories.
Commit objects recording snapshots and parent relationships.
Tag objects identifying tagged objects.

References, commonly called refs, point to commits. Branches and tags are examples.

When an agent pushes a change, it transfers the required Git objects and requests an update to a reference.

This distinction matters when designing scalable storage.

Git objects are immutable. When a file changes, Git creates a new object rather than modifying the existing object. Multiple commits can therefore share unchanged objects.

References are mutable. A branch such as refs/heads/main moves from one commit to another when a new update is accepted.

This creates different scaling requirements:

Object storage needs efficient parallel writes and reads.
Reference updates need concurrency control to prevent conflicting changes.
Fetch operations need an efficient way to identify and transfer the objects required by a client.

A scalable Git service should avoid coordinating unrelated operations unnecessarily while preserving correctness when multiple clients update the same branch.

Reduce Contention on Shared Branches

A common bottleneck appears when thousands of agents work independently but eventually push their changes into the same main branch.

Even if the agents use separate feature branches, the final merge operations still converge on one reference.

For example:

                 main
                   |
        +----------+----------+
        |          |          |
      Agent 1   Agent 2    Agent 3
        |          |          |
      Branch A  Branch B   Branch C
        |          |          |
        +----------+----------+
                   |
              Merge Queue
                   |
                  main

The merge queue becomes the coordination point.

A merge queue can ensure that changes are tested against the latest target branch before being merged. It also helps serialize updates when necessary.

However, a merge queue should not become an uncontrolled backlog.

For large agent fleets, consider:

Limiting the number of simultaneous merge candidates.
Grouping compatible changes into batches where the workflow supports it.
Prioritizing changes based on dependency and delivery requirements.
Revalidating changes when the target branch moves.
Avoiding unnecessary commits directly to the shared branch.

Not every update requires global coordination. Independent branch updates should remain independent wherever the Git hosting architecture permits it.

Improve Push Throughput With Parallel Write Processing

A Git service must accept incoming packfiles, validate the requested reference updates, store objects durably, and make accepted changes visible to readers.

A conventional architecture can create unnecessary coordination between unrelated branch updates.

For example, a service that serializes all repository writes through one shared transaction or manifest may cause pushes to different branches to wait for one another.

A more scalable design separates the storage of immutable objects from the publication of reference updates.

A simplified architecture looks like this:

              Agent Push Requests
                       |
                       v
                 Git Service
                       |
             +---------+---------+
             |                   |
             v                   v
       Object Storage       Ref Metadata
       Immutable Packs      Branch Versions
             |                   |
             +---------+---------+
                       |
                       v
                Fetch Services
                       |
                       v
                CI and Agents

The object storage layer can accept independent packfile writes. Reference metadata records which commit is associated with each branch version and which objects are required to reconstruct its history.

The important requirement is that a reference must never become visible before the objects reachable from its new commit are available.

This ordering preserves repository correctness while allowing more work to happen concurrently.

A recent example of this architectural direction is Pando, the Git backend developed by OpenAI for its monorepo. Its design coordinates updates to individual branches independently and reuses packfiles across fetches. OpenAI reported that the system reduced median push latency by nearly five times in its production environment.

This is a specific infrastructure implementation, not a configuration that can simply be enabled in an ordinary Git repository. The broader lesson is to minimize unnecessary coordination between independent branch writes.

Make Fetch Operations Cheaper

Scaling writes is only half the problem. A large number of agents and CI workers may fetch the same repository history repeatedly.

Git normally generates a packfile containing objects that the client needs but does not already have. This process can consume substantial CPU and disk I/O for large repositories.

When many clients request similar data, repeatedly generating equivalent packfiles wastes resources.

Several techniques can reduce this workload.

Reuse packfiles

A Git hosting service can cache previously generated fetch responses when requests are identical or sufficiently similar.

GitLab Gitaly provides a pack-objects cache that can reduce repeated pack generation for concurrent fetch workloads.

This is particularly useful when many CI jobs fetch the same repository state.

Use shallow clones where appropriate

A shallow clone limits the amount of history transferred.

For example:

git clone --depth 1 https://example.com/team/application.git

This is useful for CI jobs that only need the latest source snapshot.

However, shallow clones are not appropriate for every workflow. Operations that require full history, certain merge strategies, or historical analysis may need additional commits.

Use partial clones

Partial clones allow clients to retrieve selected objects instead of downloading the entire repository immediately.

For example:

git clone --filter=blob:none https://example.com/team/application.git

This initially avoids downloading file contents and retrieves missing blobs when they are needed.

Partial clones can reduce transfer volume, particularly in repositories with large historical files. They also introduce additional network requests when missing objects are accessed, so the workload should be measured before adoption.

Reduce unnecessary CI fetches

Review whether every pipeline needs a full checkout. Avoid repeated clones when a reusable workspace or artifact can safely serve the same purpose.

The objective is not to eliminate fetches but to avoid repeatedly transferring and generating data that clients already have.

Manage Packfile Compaction and Repository Maintenance

Git stores objects individually as loose objects or together in packfiles.

Packfiles reduce storage overhead and improve transfer efficiency by compressing related objects. However, frequent pushes can produce many small packs that eventually need to be reorganized.

Git maintenance operations such as repacking and garbage collection help manage this storage.

For example:

git gc

This runs Git's housekeeping process.

For a repository receiving thousands of writes, running expensive maintenance operations at the wrong time can compete with active pushes and fetches for CPU and disk I/O.

A better strategy is to schedule maintenance according to repository activity and monitor its resource consumption.

Useful considerations include:

Avoid overlapping expensive maintenance jobs unnecessarily.
Monitor loose-object growth.
Track packfile count and size.
Use incremental maintenance where appropriate.
Monitor storage latency during repacking.
Verify that unreachable objects are retained or removed according to the repository's recovery requirements.

For very high-volume systems, packfile generation and compaction may need to become separate background operations rather than blocking the normal request path.

The storage architecture should also support efficient fetches while compaction is running.

Apply Backpressure Instead of Allowing Unlimited Concurrency

Allowing every incoming request to start immediately can overwhelm a Git server.

A large number of simultaneous pushes and fetches may exhaust CPU, disk bandwidth, memory, or available connections.

Concurrency controls provide backpressure by limiting how much work the server accepts at one time.

For example, GitLab Gitaly supports limits for RPC operations and pack-object generation. Requests can be queued or rejected when configured limits are reached.

This helps protect the storage service, but limits must be chosen carefully. An excessively low limit can make a healthy repository unnecessarily slow.

A practical design should consider:

Per-repository concurrency.
Per-user or per-agent request limits.
Maximum queue length.
Queue waiting time.
Request timeouts.
Separate limits for expensive operations.
Priority for critical infrastructure requests.

When a request is rejected, the client should use bounded retries with exponential backoff and jitter.

For example:

import random
import time

def retry_push(push_operation, max_attempts=5):
    for attempt in range(max_attempts):
        try:
            return push_operation()
        except TemporaryGitError:
            if attempt == max_attempts - 1:
                raise

            delay = min(2 ** attempt, 30)
            time.sleep(delay + random.uniform(0, 1))

TemporaryGitError is a placeholder for the application's actual retryable error type. The push operation must also be safe to retry. A failed connection does not always prove that the server rejected the update; the client may need to inspect the remote reference before attempting the operation again.

Unbounded retries are particularly dangerous in agent fleets because they can turn a temporary slowdown into a request storm.

Monitor Repository Performance

A high-volume Git service needs monitoring that distinguishes write problems from read problems and maintenance activity.

Useful metrics include:

Metric	What it reveals
Push latency	Time required to complete a push
Fetch latency	Time required to retrieve repository data
Push success rate	Reliability of write operations
Packfile generation time	CPU and I/O cost of preparing fetch responses
Packfile cache hit rate	Effectiveness of fetch-response reuse
Ref update conflicts	Contention on shared branches
Queue depth	Requests waiting for available capacity
Storage IOPS and latency	Disk pressure
Repository growth	Long-term storage requirements
CI fetch volume	Downstream read amplification

Track latency percentiles rather than relying only on averages. A service may have acceptable average push times while a small percentage of agents experience very slow operations.

Correlate Git metrics with CI activity and agent concurrency. This helps identify whether a slowdown is caused by excessive writes, repeated fetches, merge contention, or storage maintenance.

Common Mistakes When Scaling Agent-Heavy Git Workloads

Mistake	Why it causes problems	Better approach
Sending every agent directly to main	Creates unnecessary reference contention	Use independent branches and controlled merging
Increasing concurrency without measurement	Can overwhelm storage and CPU	Apply monitored, workload-aware limits
Running full clones for every CI job	Repeatedly transfers unnecessary history	Evaluate shallow and partial clones
Running maintenance during peak traffic	Competes with active Git operations	Schedule and monitor repository maintenance
Retrying failed pushes immediately	Creates additional load during an outage	Use bounded retries with backoff and jitter
Treating push latency as the only metric	Ignores fetch amplification and storage bottlenecks	Monitor the complete Git workload
Assuming more replicas solve write contention	Replication does not automatically remove coordination	Design independent write paths and consistent ref publication

Conclusion

Thousands of agents pushing changes into a shared Git repository create a different workload from traditional human-driven development. Frequent checkpoints increase write traffic, CI pipelines multiply fetch requests, and merge queues concentrate updates on shared references.

Keeping repositories responsive requires a balanced approach. Independent branch writes should avoid unnecessary global coordination, fetch responses should be reused where possible, and repository maintenance must be planned around active workloads.

Concurrency controls, efficient cloning strategies, packfile caching, and detailed observability help manage the immediate infrastructure pressure. At larger scales, Git hosting systems may also need storage architectures that separate immutable object writes from reference coordination.

The objective is not simply to accept more pushes. It is to make sure that every accepted change becomes available reliably, that readers can retrieve repository history efficiently, and that one busy agent or branch does not slow down the entire engineering workflow.

Git scalabilityGit repositoriesAI agentsconcurrent pushesmerge queuesGit infrastructurerepository performance

Comments

Join the conversation! Your thoughts help the community grow.

Keeping Git Repositories Responsive When Thousands of Agents Push Changes

Saurav Kumar

8h2.4k00100Article

Understand Why Thousands of Agents Change Git Workloads

Traditional development usually produces an uneven workload. Developers edit code, run tests, and push changes when a task is ready.

Agents often follow shorter execution cycles:

Retrieve the latest repository state.
Modify a set of files.
Commit a checkpoint.
Push the branch.
Trigger automated validation.
Fetch updates and continue working.

When thousands of agents follow this pattern, repository activity becomes continuous rather than occasional.

There are four major sources of pressure.

Concurrent writes

Each push transfers Git objects and requests updates to one or more references. A repository receiving many pushes must process these writes while maintaining correct branch state.

Read amplification

One push can trigger several downstream reads. CI pipelines, code scanning, test environments, and other agents may all fetch the same commit.

A single branch update can therefore generate many more fetch operations than push operations.

Reference contention

Repository maintenance

These workloads need to be considered together. Optimizing push latency while allowing fetch traffic or repository maintenance to overwhelm storage will not solve the overall problem.

Understand the Difference Between Git Objects and References

A Git repository contains two important categories of data: objects and references.

Objects store the repository's content and history. They include:

Blob objects containing file content.
Tree objects describing directories.
Commit objects recording snapshots and parent relationships.
Tag objects identifying tagged objects.

References, commonly called refs, point to commits. Branches and tags are examples.

When an agent pushes a change, it transfers the required Git objects and requests an update to a reference.

This distinction matters when designing scalable storage.

Git objects are immutable. When a file changes, Git creates a new object rather than modifying the existing object. Multiple commits can therefore share unchanged objects.

References are mutable. A branch such as refs/heads/main moves from one commit to another when a new update is accepted.

This creates different scaling requirements:

Object storage needs efficient parallel writes and reads.
Reference updates need concurrency control to prevent conflicting changes.
Fetch operations need an efficient way to identify and transfer the objects required by a client.

A scalable Git service should avoid coordinating unrelated operations unnecessarily while preserving correctness when multiple clients update the same branch.

Reduce Contention on Shared Branches

A common bottleneck appears when thousands of agents work independently but eventually push their changes into the same main branch.

Even if the agents use separate feature branches, the final merge operations still converge on one reference.

For example:

                 main
                   |
        +----------+----------+
        |          |          |
      Agent 1   Agent 2    Agent 3
        |          |          |
      Branch A  Branch B   Branch C
        |          |          |
        +----------+----------+
                   |
              Merge Queue
                   |
                  main

The merge queue becomes the coordination point.

A merge queue can ensure that changes are tested against the latest target branch before being merged. It also helps serialize updates when necessary.

However, a merge queue should not become an uncontrolled backlog.

For large agent fleets, consider:

Limiting the number of simultaneous merge candidates.
Grouping compatible changes into batches where the workflow supports it.
Prioritizing changes based on dependency and delivery requirements.
Revalidating changes when the target branch moves.
Avoiding unnecessary commits directly to the shared branch.

Not every update requires global coordination. Independent branch updates should remain independent wherever the Git hosting architecture permits it.

Improve Push Throughput With Parallel Write Processing

A Git service must accept incoming packfiles, validate the requested reference updates, store objects durably, and make accepted changes visible to readers.

A conventional architecture can create unnecessary coordination between unrelated branch updates.

For example, a service that serializes all repository writes through one shared transaction or manifest may cause pushes to different branches to wait for one another.

A more scalable design separates the storage of immutable objects from the publication of reference updates.

A simplified architecture looks like this:

              Agent Push Requests
                       |
                       v
                 Git Service
                       |
             +---------+---------+
             |                   |
             v                   v
       Object Storage       Ref Metadata
       Immutable Packs      Branch Versions
             |                   |
             +---------+---------+
                       |
                       v
                Fetch Services
                       |
                       v
                CI and Agents

The important requirement is that a reference must never become visible before the objects reachable from its new commit are available.

This ordering preserves repository correctness while allowing more work to happen concurrently.

Make Fetch Operations Cheaper

Scaling writes is only half the problem. A large number of agents and CI workers may fetch the same repository history repeatedly.

Git normally generates a packfile containing objects that the client needs but does not already have. This process can consume substantial CPU and disk I/O for large repositories.

When many clients request similar data, repeatedly generating equivalent packfiles wastes resources.

Several techniques can reduce this workload.

Reuse packfiles

A Git hosting service can cache previously generated fetch responses when requests are identical or sufficiently similar.

GitLab Gitaly provides a pack-objects cache that can reduce repeated pack generation for concurrent fetch workloads.

This is particularly useful when many CI jobs fetch the same repository state.

Use shallow clones where appropriate

A shallow clone limits the amount of history transferred.

For example:

git clone --depth 1 https://example.com/team/application.git

This is useful for CI jobs that only need the latest source snapshot.

However, shallow clones are not appropriate for every workflow. Operations that require full history, certain merge strategies, or historical analysis may need additional commits.

Use partial clones

Partial clones allow clients to retrieve selected objects instead of downloading the entire repository immediately.

For example:

git clone --filter=blob:none https://example.com/team/application.git

This initially avoids downloading file contents and retrieves missing blobs when they are needed.

Reduce unnecessary CI fetches

Review whether every pipeline needs a full checkout. Avoid repeated clones when a reusable workspace or artifact can safely serve the same purpose.

The objective is not to eliminate fetches but to avoid repeatedly transferring and generating data that clients already have.

Manage Packfile Compaction and Repository Maintenance

Git stores objects individually as loose objects or together in packfiles.

Packfiles reduce storage overhead and improve transfer efficiency by compressing related objects. However, frequent pushes can produce many small packs that eventually need to be reorganized.

Git maintenance operations such as repacking and garbage collection help manage this storage.

For example:

git gc

This runs Git's housekeeping process.

For a repository receiving thousands of writes, running expensive maintenance operations at the wrong time can compete with active pushes and fetches for CPU and disk I/O.

A better strategy is to schedule maintenance according to repository activity and monitor its resource consumption.

Useful considerations include:

Avoid overlapping expensive maintenance jobs unnecessarily.
Monitor loose-object growth.
Track packfile count and size.
Use incremental maintenance where appropriate.
Monitor storage latency during repacking.
Verify that unreachable objects are retained or removed according to the repository's recovery requirements.

For very high-volume systems, packfile generation and compaction may need to become separate background operations rather than blocking the normal request path.

The storage architecture should also support efficient fetches while compaction is running.

Apply Backpressure Instead of Allowing Unlimited Concurrency

Allowing every incoming request to start immediately can overwhelm a Git server.

A large number of simultaneous pushes and fetches may exhaust CPU, disk bandwidth, memory, or available connections.

Concurrency controls provide backpressure by limiting how much work the server accepts at one time.

For example, GitLab Gitaly supports limits for RPC operations and pack-object generation. Requests can be queued or rejected when configured limits are reached.

This helps protect the storage service, but limits must be chosen carefully. An excessively low limit can make a healthy repository unnecessarily slow.

A practical design should consider:

Per-repository concurrency.
Per-user or per-agent request limits.
Maximum queue length.
Queue waiting time.
Request timeouts.
Separate limits for expensive operations.
Priority for critical infrastructure requests.

When a request is rejected, the client should use bounded retries with exponential backoff and jitter.

For example:

import random
import time

def retry_push(push_operation, max_attempts=5):
    for attempt in range(max_attempts):
        try:
            return push_operation()
        except TemporaryGitError:
            if attempt == max_attempts - 1:
                raise

            delay = min(2 ** attempt, 30)
            time.sleep(delay + random.uniform(0, 1))

Unbounded retries are particularly dangerous in agent fleets because they can turn a temporary slowdown into a request storm.

Monitor Repository Performance

A high-volume Git service needs monitoring that distinguishes write problems from read problems and maintenance activity.

Useful metrics include:

Metric	What it reveals
Push latency	Time required to complete a push
Fetch latency	Time required to retrieve repository data
Push success rate	Reliability of write operations
Packfile generation time	CPU and I/O cost of preparing fetch responses
Packfile cache hit rate	Effectiveness of fetch-response reuse
Ref update conflicts	Contention on shared branches
Queue depth	Requests waiting for available capacity
Storage IOPS and latency	Disk pressure
Repository growth	Long-term storage requirements
CI fetch volume	Downstream read amplification

Track latency percentiles rather than relying only on averages. A service may have acceptable average push times while a small percentage of agents experience very slow operations.

Correlate Git metrics with CI activity and agent concurrency. This helps identify whether a slowdown is caused by excessive writes, repeated fetches, merge contention, or storage maintenance.

Common Mistakes When Scaling Agent-Heavy Git Workloads

Mistake	Why it causes problems	Better approach
Sending every agent directly to main	Creates unnecessary reference contention	Use independent branches and controlled merging
Increasing concurrency without measurement	Can overwhelm storage and CPU	Apply monitored, workload-aware limits
Running full clones for every CI job	Repeatedly transfers unnecessary history	Evaluate shallow and partial clones
Running maintenance during peak traffic	Competes with active Git operations	Schedule and monitor repository maintenance
Retrying failed pushes immediately	Creates additional load during an outage	Use bounded retries with backoff and jitter
Treating push latency as the only metric	Ignores fetch amplification and storage bottlenecks	Monitor the complete Git workload
Assuming more replicas solve write contention	Replication does not automatically remove coordination	Design independent write paths and consistent ref publication

Conclusion

Git scalabilityGit repositoriesAI agentsconcurrent pushesmerge queuesGit infrastructurerepository performance

Comments

Join the conversation! Your thoughts help the community grow.