Introduction
AI coding agents can work on a task for much longer than a traditional chat assistant. They inspect repositories, read source files, execute commands, analyze test failures, modify code, and repeat the process until the task is complete.
The problem is that every interaction adds more information to the agent's working context.
Eventually, the agent has to decide what historical information should remain available and what can be compressed or removed. This process is generally called context compaction.
Compaction sounds straightforward: make the context smaller.
In practice, the strategy used to reduce context can significantly affect agent quality, latency, tool usage, and the amount of work the model has to repeat.
For example, an agent could compact:
After every fixed number of tool calls
When the context reaches a token threshold
Only when the model reports context pressure
At natural task boundaries
By summarizing everything
By preserving structured state and discarding raw history
These approaches are not equivalent.
This article presents a practical framework for benchmarking context compaction strategies in AI coding agents and explains how to measure whether a strategy actually improves the overall workload.
Why Context Compaction Matters
Consider an agent working on a moderately complex .NET application.
During one task, it might produce:
User request
|
+--> Repository search
|
+--> Project structure
|
+--> Read source files
|
+--> Read tests
|
+--> Modify implementation
|
+--> Build
|
+--> Test
|
+--> Analyze failure
|
+--> Modify again
|
+--> Test again
Every step can add context.
A long session may eventually contain:
Task
+ Conversation
+ Source snippets
+ Tool calls
+ Tool results
+ Compiler output
+ Test output
+ Failed approaches
+ Architectural decisions
+ Generated code
Keeping all of this indefinitely is expensive and can make the context difficult to manage.
But removing too much information creates another problem.
The agent may forget:
Why a design decision was made
Which files were already modified
Which approach failed
What tests are still failing
Which repository constraints matter
What the user originally requested
Therefore, context compaction is an optimization problem rather than a simple token-reduction exercise.
What Is Context Compaction?
Context compaction transforms a large conversation or agent history into a smaller representation.
For example:
Before:
Task
|
+-- User request
+-- Search results
+-- File A
+-- File B
+-- Build output
+-- Test output
+-- Failed implementation
+-- New implementation
+-- Final test output
After:
Task
|
+-- Objective
+-- Modified files
+-- Important decisions
+-- Known failures
+-- Current state
+-- Remaining work
The second representation is smaller, but ideally it retains the information necessary to continue the task.
A successful compaction strategy should reduce unnecessary context without forcing the agent to rediscover important information.
The Main Compaction Strategies
There are several strategies worth benchmarking.
Fixed-Interval Compaction
The simplest approach is to compact after a fixed number of model or tool calls.
For example:
Every 20 tool calls
|
v
Compact context
|
v
Continue task
This is easy to implement but does not consider the actual content of the context.
A task may need compaction after 8 large tool results while another task may remain efficient after 30 small interactions.
Token-Threshold Compaction
Another approach is to compact when context reaches a configured size.
Context size
|
v
Threshold reached?
|
Yes
|
v
Compact
This is more adaptive than a fixed call count.
However, token count alone does not tell you how valuable the information is.
Two contexts with the same number of tokens can have very different information density.
Event-Driven Compaction
A more intelligent strategy is to compact when the agent reaches a meaningful workflow event.
For example:
Implementation complete
|
v
Run tests
|
v
Compaction checkpoint
|
v
Analyze failures
This can preserve important boundaries between phases of work.
Structured-State Compaction
Instead of asking the model to summarize the entire conversation, the system maintains explicit task state.
For example:
{
"objective": "Add retry support to PaymentService",
"modifiedFiles": [
"PaymentService.cs",
"PaymentServiceTests.cs"
],
"decisions": [
"Use bounded exponential backoff"
],
"knownFailures": [
"Timeout test still fails"
],
"remainingWork": [
"Fix timeout handling"
]
}
This approach can be more deterministic because important information does not depend entirely on a free-form summary.
A Benchmark Should Compare Strategies, Not Just Token Counts
A common mistake is to define success as:
Smaller context = Better strategy
That is incomplete.
Suppose Strategy A reduces context by 60% but causes the agent to reread files repeatedly.
Strategy B reduces context by only 35% but allows the agent to complete the task with fewer tool calls.
Strategy B may be better overall.
A useful benchmark therefore measures:
| Metric | What it tells you |
|---|
| Context tokens | How much context is processed |
| Compaction count | How frequently compaction occurs |
| Tool calls | How much additional work the agent performs |
| Model calls | Overall inference activity |
| Task success | Whether the agent actually completes the task |
| Rediscovery rate | How often information must be obtained again |
| Latency | End-to-end completion time |
| Cache reuse | How much prior context remains reusable |
| Error rate | Whether compaction causes incorrect decisions |
| Cost estimate | Relative resource consumption |
The benchmark should evaluate the complete workload.
Designing a Representative Task Set
Do not benchmark compaction using only one coding task.
A useful test suite should contain several workload categories.
Small Tasks
Examples:
Rename a method
Add a validation rule
Fix a simple unit test
These determine whether compaction is unnecessary overhead.
Medium Tasks
Examples:
Add a service
Modify an API endpoint
Update persistence logic
Add integration tests
These create enough history for compaction to matter.
Large Tasks
Examples:
Refactor a subsystem
Migrate an API
Debug multiple test failures
Implement a cross-layer feature
These are useful for measuring long-session behavior.
Failure-Heavy Tasks
Include tasks where the first implementation does not work.
For example:
Implement
|
v
Build fails
|
v
Fix
|
v
Tests fail
|
v
Investigate
|
v
Fix again
These workloads are particularly useful because the agent needs to remember previous failures and decisions.
Establishing a Baseline
Before testing compaction, create a baseline with no intentional compaction.
Strategy 0
---------
No compaction
Full available history
Record:
Total context tokens
Total model calls
Total tool calls
Completion time
Task success
This baseline provides a reference point.
Without it, you cannot determine whether compaction actually improved the workload.
Building a Simple Benchmark Harness
A benchmark runner can represent each strategy explicitly.
public interface ICompactionStrategy
{
bool ShouldCompact(AgentContext context);
AgentContext Compact(AgentContext context);
}
A token-based strategy could look like:
public sealed class TokenThresholdStrategy
: ICompactionStrategy
{
private readonly int _threshold;
public TokenThresholdStrategy(int threshold)
{
_threshold = threshold;
}
public bool ShouldCompact(AgentContext context)
{
return context.TokenCount >= _threshold;
}
public AgentContext Compact(AgentContext context)
{
return context.CreateSummary();
}
}
A fixed-call strategy could use tool-call count instead:
public sealed class ToolCallStrategy
: ICompactionStrategy
{
private readonly int _interval;
public ToolCallStrategy(int interval)
{
_interval = interval;
}
public bool ShouldCompact(AgentContext context)
{
return context.ToolCallCount > 0 &&
context.ToolCallCount % _interval == 0;
}
public AgentContext Compact(AgentContext context)
{
return context.CreateSummary();
}
}
The important part is not the implementation itself.
It is the ability to run the same task workload against different strategies.
Keep the Benchmark Controlled
A useful benchmark needs to control variables that could otherwise distort the result.
Keep the following consistent where possible:
The only major variable should be the compaction strategy.
For example:
Repository
|
+---- Strategy A
|
+---- Strategy B
|
+---- Strategy C
|
+---- Strategy D
Each strategy should receive an equivalent starting state.
Measuring Rediscovery
One of the most useful metrics is rediscovery work.
Suppose an agent forgets that PaymentService.cs contains a specific implementation detail.
It may call:
Search
|
v
Read file
|
v
Analyze
The agent has now spent additional work recovering information that existed before compaction.
You can classify tool calls as:
Necessary work
Rediscovery work
Duplicate work
A simple conceptual metric is:
Rediscovery Rate =
Rediscovery Tool Calls
----------------------
Total Tool Calls
The exact classification should be defined consistently across the benchmark.
A high rediscovery rate is a warning sign that the compaction strategy is removing information the agent still needs.
Measuring Task Quality
A strategy that saves tokens but produces incorrect code is not an optimization.
Evaluate the final result using objective checks such as:
Build succeeds
Tests pass
Expected files changed
Required behavior exists
No prohibited changes introduced
For coding tasks, automated tests are particularly valuable.
For example:
dotnet build
dotnet test
You can then record:
Build: PASS
Tests: 42/42
Task: PASS
If the task requires human judgment, supplement automated evaluation with a structured review rubric.
Measuring Compaction Loss
A useful benchmark should identify what information disappears during compaction.
Classify lost information into categories:
| Information | Impact if lost |
|---|
| Original requirement | Very high |
| Architecture decision | High |
| Modified files | High |
| Known failure | High |
| Test result | High |
| Temporary search result | Low |
| Duplicate logs | Low |
| Old compiler output | Usually low |
This helps explain why one compaction strategy performs better than another.
Example Benchmark Output
A benchmark report might look like this:
| Strategy | Context Reduction | Tool Calls | Task Success | Rediscovery | Latency |
|---|
| No compaction | 0% | Baseline | Baseline | Baseline | Baseline |
| Fixed interval | Example | Example | Example | Example | Example |
| Token threshold | Example | Example | Example | Example | Example |
| Event-driven | Example | Example | Example | Example | Example |
| Structured state | Example | Example | Example | Example | Example |
The values should come from actual benchmark execution.
Do not use invented numbers to claim that one strategy is universally superior.
Testing Cache Effects
Context compaction can also affect caching behavior.
Consider:
Before compaction:
[A][B][C][D][E]
After compaction:
[S][D][E]
The new context no longer has the same prefix.
Depending on the serving system, previously processed context may therefore become less reusable.
This means your benchmark should ideally record cache-related telemetry when it is available.
Useful measurements include:
Cache hit rate
Cache miss rate
Cached prefix length
Context size
Compaction events
Model switches
This matters because a strategy that reduces token count may simultaneously reduce cache reuse.
Compaction and Model Switching
Model switching should be treated as another benchmark variable.
For example:
Task
|
+--> Model A
|
+--> Compaction
|
+--> Model B
Now two changes have occurred:
Context was compacted.
The model changed.
If performance changes, it becomes difficult to determine which event caused the effect.
Therefore, benchmark these scenarios separately:
A. Same model + no compaction
B. Same model + compaction
C. Model switch + no compaction
D. Model switch + compaction
This produces a much cleaner comparison.
Common Benchmarking Mistakes
Comparing Different Tasks
A strategy may appear better simply because it received an easier workload.
Use the same task corpus.
Measuring Only Tokens
Lower token usage does not guarantee lower latency or better task completion.
Ignoring Tool Calls
If compaction causes the agent to repeatedly search and read files, model-token savings may be offset by additional tool activity.
Using Only Short Tasks
Compaction strategies have little opportunity to differentiate themselves when the entire task fits comfortably inside the initial context.
Ignoring Failed Attempts
Long debugging workflows are often where context management becomes most important.
Changing Multiple Variables at Once
If model, prompt, tools, repository, and compaction strategy all change simultaneously, the benchmark cannot explain the result.
Using Synthetic Tasks Only
Synthetic workloads are useful, but real repository tasks often contain more realistic tool-call patterns and failure modes.
Best Practices for Context Compaction
Preserve Structured State
Keep important information outside the raw conversation:
Task
Modified files
Decisions
Known failures
Current step
Remaining work
Compact at Meaningful Boundaries
A completed investigation phase is often a better compaction point than an arbitrary tool-call count.
Preserve Failure Information
An agent should not rediscover an approach that has already been proven incorrect.
Remove Low-Value Noise
Large repetitive logs can usually be reduced to the relevant error and diagnostic information.
Measure Rediscovery
If the agent repeatedly searches for information that was removed, the compaction strategy is too aggressive.
Monitor Cache Behavior
Context reduction and cache efficiency are related but different objectives.
Benchmark Long Sessions
Use tasks that require enough interactions for context management to become meaningful.
A Practical Evaluation Framework
A useful evaluation can score each strategy across five dimensions:
Context Efficiency
+
Task Quality
+
Tool Efficiency
+
Latency
+
Operational Cost
For example:
Strategy Score
=
Quality
+
Efficiency
+
Reliability
Avoid creating a universal weighting without understanding the application's priorities.
For a coding assistant, task correctness may matter more than a modest reduction in context processing.
For a high-volume autonomous agent, infrastructure efficiency may have greater importance.
The benchmark should reflect the actual production objective.
Frequently Asked Questions
Is context compaction always beneficial?
No. Compaction introduces processing and can remove information the agent needs later. It is beneficial when the retained context has become less valuable than the cost of carrying it forward.
What is the best compaction strategy?
There is no universal best strategy. The right approach depends on task duration, context size, tool behavior, model characteristics, and the cost of rediscovery.
Should I compact after a fixed number of tool calls?
It can be a useful baseline, but it should not automatically be considered the production strategy. Token thresholds or event-driven approaches can adapt better to workload differences.
Should important information be stored outside the conversation?
For long-running agents, yes. Explicit structured state can make critical information more reliable and easier to preserve through compaction.
How do I know whether compaction is too aggressive?
Look for increasing rediscovery, repeated tool calls, forgotten requirements, incorrect decisions, or declining task success after compaction.
Should cache hit rate be part of the benchmark?
Yes, when the serving infrastructure exposes meaningful cache telemetry. Context compaction can change the request prefix and therefore affect cache reuse.
Conclusion
Context compaction is becoming an important engineering concern as AI coding agents move from short conversations to long-running autonomous workflows.
The goal is not simply to minimize the number of tokens in a request. A useful strategy must preserve the information needed for correct decisions while reducing unnecessary historical context.
That is why benchmarking should measure the complete workload: context size, compaction frequency, model calls, tool calls, rediscovery, cache behavior, latency, and task success.
A practical benchmark should begin with a no-compaction baseline and then compare fixed-interval, token-threshold, event-driven, and structured-state approaches using the same repository and task corpus.
The most important metric is ultimately not how small the context becomes. It is whether the agent can continue solving the task correctly without repeatedly rediscovering information that it already learned.
That is the real measure of an effective context compaction strategy.