AI agents can do much more than a traditional API call.
A normal application might send one request to a model and receive one response. An AI agent can take a goal, decide what to do, call tools, inspect results, make another decision, and continue until it reaches an outcome.
That flexibility introduces another engineering problem - cost.
An agent that looks inexpensive during development can become expensive when hundreds or thousands of users start using it. The reason is simple: the cost is not limited to the final model response. Every model call, tool execution, retry, database query, search operation, and additional context can contribute to the total cost.
Before putting an AI agent into production, developers should measure the complete execution path.
Why AI Agent Cost Is Different
A traditional AI request often looks like:
User Request
|
v
LLM
|
v
Response
An agent can look like:
User Request
|
v
Agent
|
+----> Model Call
|
+----> Tool Call
|
+----> Model Call
|
+----> Database
|
+----> Model Call
|
+----> Tool Call
|
v
Final Response
Each additional step can consume resources.
For example, an agent may need three model calls to complete a task that a developer expected to require one.
If one execution costs:
Model calls $0.03
Tool execution $0.01
Database/search $0.01
Infrastructure $0.005
--------------------------
Total $0.055
the application needs to understand that full cost before estimating production economics.
Start With Cost Per Agent Run
The first useful metric is cost per completed agent run.
A simple model is:
Total Agent Cost
=
Model Cost
+
Tool Cost
+
Infrastructure Cost
+
Storage Cost
+
Other Service Costs
The exact components depend on the architecture.
For a simple application, the model may dominate the bill.
For a tool-heavy agent, external services and infrastructure can become important as well.
The key is to measure the entire workflow instead of looking only at the model invoice.
Count Every Model Call
One of the easiest mistakes is measuring only the final response.
Consider this agent:
1. Understand request
2. Search documentation
3. Analyze search results
4. Call database
5. Validate result
6. Generate final response
The agent might make four or five model calls.
A useful execution record could look like:
{
"runId": "run-1024",
"modelCalls": 4,
"inputTokens": 18200,
"outputTokens": 3200,
"toolCalls": 3,
"durationMs": 8400
}
This makes the execution measurable.
Without this information, it is difficult to explain why one request costs significantly more than another.
Measure Input and Output Tokens
Token usage is one of the main variables in model cost.
An agent can consume tokens through:
User prompts
System instructions
Conversation history
Tool definitions
Tool results
Retrieved documents
Previous agent steps
Final responses
For example:
System instructions 1,500 tokens
User request 300 tokens
Conversation history 4,000 tokens
Retrieved documents 8,000 tokens
Tool results 3,000 tokens
---------------------------------------
Input 16,800 tokens
If this context is sent repeatedly across multiple model calls, the total token usage can grow quickly.
That is why measuring tokens per individual model call is important.
Track Cost Per Step
Instead of recording only the final cost, create a breakdown.
For example:
Step | Operation | Input Tokens | Output Tokens | Duration |
|---|
1 | Planning | 2,000 | 500 | 900 ms |
2 | Search analysis | 5,000 | 700 | 1.4 s |
3 | Tool decision | 4,000 | 300 | 800 ms |
4 | Final response | 6,000 | 900 | 1.7 s |
This gives developers a much clearer picture.
Suppose step 2 consumes most of the tokens.
The optimization target is now obvious:
Large retrieved context
|
v
Reduce context
|
v
Lower token usage
Without step-level measurement, developers may optimize the wrong part of the system.
Calculate Cost Per Successful Task
Cost per request is useful, but cost per successful task is often more meaningful.
Imagine:
1,000 agent runs
and:
850 successful
150 failed
If the total cost is $80:
Cost per run = $0.08
But the cost per successful task is:
$80 / 850 = $0.094
The difference becomes more important when failures trigger retries or additional processing.
A production system should therefore track both:
Cost per run
Cost per successful outcome
Retries Can Increase Cost Quickly
Agents often retry failed operations.
For example:
Model call
|
v
Tool fails
|
v
Retry
|
v
Tool fails
|
v
Retry
|
v
Success
One user request has now created several operations.
A simple retry policy might look like:
for (int attempt = 1; attempt <= 3; attempt++)
{
try
{
return await ExecuteAgentStepAsync();
}
catch when (attempt < 3)
{
await Task.Delay(1000);
}
}
throw new InvalidOperationException("Agent step failed.");
Retries should be measured separately.
Useful metrics include:
Retry count
Retry rate
Cost of retries
Successful retry rate
If a tool frequently fails and the agent repeatedly retries it, the cost problem may actually be a reliability problem.
Measure Tool Calls Too
Tools can have their own cost.
An agent might call:
Search API
Database
Cloud function
External API
File processing service
For each tool, measure:
Call count
Execution time
Failure rate
Cost
Returned data size
For example:
Tool | Calls | Avg Duration | Failure Rate |
|---|
Search | 3 | 450 ms | 2% |
Database | 2 | 120 ms | 0.5% |
External API | 1 | 1.8 s | 4% |
This can reveal expensive or unreliable parts of the agent workflow.
Large Tool Results Can Become Expensive
Consider a search tool that returns thousands of records.
{
"results": [
"...",
"...",
"... many records ..."
]
}
The agent may not need all of them.
Sending unnecessary results back to the model increases context size.
A better design is to limit and summarize tool output before passing it to the model.
For example:
Database
|
v
10,000 records
|
v
Filter
|
v
100 relevant records
|
v
Agent
This can reduce both latency and token consumption.
Measure Context Growth
Agent conversations can become expensive when every previous step is included in subsequent model calls.
For example:
Call 1
2,000 tokens
Call 2
4,000 tokens
Call 3
8,000 tokens
Call 4
12,000 tokens
The agent's context is growing.
Track the size of the context sent to each model call.
A useful metric is:
Average input tokens per step
Also track:
Maximum input tokens
P95 input tokens
Total tokens per run
The average alone may hide expensive outliers.
Measure P95 and P99 Cost
Production traffic is rarely uniform.
Most requests might cost:
$0.02 - $0.05
while a small number might cost:
$0.50+
Those expensive executions matter.
Track:
Average cost
Median cost
P95 cost
P99 cost
Maximum cost
For example:
Metric | Value |
|---|
Average | $0.06 |
Median | $0.04 |
P95 | $0.13 |
P99 | $0.42 |
Maximum | $1.10 |
This gives a much better understanding of production risk than the average alone.
Build a Cost Budget
Before production, define an expected cost budget.
For example:
Target cost per successful task: $0.10
Maximum acceptable cost: $0.25
The exact values depend on the application.
The important point is that the budget should be explicit.
Then the agent can enforce limits.
For example:
if (estimatedCost > maxCost)
{
throw new InvalidOperationException(
"Agent execution exceeded the configured cost limit.");
}
A real implementation should calculate the estimate from the actual model and tool usage rather than relying on a fixed placeholder.
Set Agent Execution Limits
Cost control should not depend entirely on monitoring.
The agent itself can have limits such as:
Maximum model calls
Maximum tool calls
Maximum execution time
Maximum retry count
Maximum context size
Maximum workflow depth
For example:
const int maxToolCalls = 5;
if (toolCallCount >= maxToolCalls)
{
return "The task could not be completed within the execution limit.";
}
These controls protect against unexpected loops.
Detect Agent Loops
An agent can sometimes repeatedly perform the same action.
For example:
Search
|
v
Analyze
|
v
Search
|
v
Analyze
|
v
Search
|
v
Analyze
This creates unnecessary cost.
Track repeated actions:
var toolHistory = new HashSet<string>();
if (!toolHistory.Add(toolName + ":" + argumentsHash))
{
throw new InvalidOperationException(
"Repeated tool invocation detected.");
}
The exact implementation depends on the agent framework, but the principle is useful:
An agent should have a clear stopping condition.
Measure Cost During Testing
Do not wait until production to calculate cost.
Create representative test scenarios.
For example:
Scenario A - Simple question
Scenario B - Database lookup
Scenario C - Multi-step research
Scenario D - Tool failure
Scenario E - Large context
Scenario F - Complex task
Then measure:
Scenario | Model Calls | Tool Calls | Total Tokens | Cost |
|---|
Simple | 1 | 0 | 2,000 | $0.02 |
Database | 2 | 1 | 5,000 | $0.05 |
Research | 5 | 4 | 18,000 | $0.18 |
Failure | 6 | 5 | 22,000 | $0.24 |
These values are illustrative. Production costs should come from the actual model pricing and measured workload.
Load Testing Changes the Picture
A single agent execution may look inexpensive.
Production traffic can change the economics.
Suppose:
Cost per successful task = $0.08
At:
100 tasks/day
the model workload is:
100 × $0.08 = $8/day
At:
100,000 tasks/day
the same unit cost becomes:
100,000 × $0.08 = $8,000/day
The arithmetic is simple, but it demonstrates why unit economics should be understood before launch.
If the agent becomes more expensive under high concurrency, infrastructure costs also need to be included.
Separate Fixed and Variable Costs
An AI agent can have both.
Fixed Costs
Examples include:
Hosting
Monitoring
Databases
Background services
Variable Costs
Examples include:
A simple production model is:
Monthly Cost
=
Fixed Infrastructure
+
Model Usage
+
Tool Usage
+
Storage
+
Observability
This makes capacity planning easier.
Track Cost by Customer or Feature
If an application serves multiple customers, aggregate cost alone is not enough.
Track usage by appropriate dimensions:
Customer
Project
Feature
Agent
Environment
Model
For example:
{
"customerId": "customer-42",
"agent": "support-agent",
"modelCalls": 5,
"toolCalls": 3,
"inputTokens": 12000,
"outputTokens": 1400
}
Be careful not to put sensitive user information into telemetry.
Use internal identifiers and follow the application's data-handling policies.
Add Cost Observability
An agent should expose useful telemetry.
For each execution, capture information such as:
Run ID
Agent name
Model
Model calls
Input tokens
Output tokens
Tool calls
Retries
Duration
Status
Estimated cost
A simplified C# record could be:
public sealed record AgentUsage(
string RunId,
int ModelCalls,
long InputTokens,
long OutputTokens,
int ToolCalls,
int RetryCount,
long DurationMs,
bool Succeeded);
This gives the monitoring system structured information rather than relying on application log messages.
Use a Cost Calculation Layer
Do not spread cost calculations throughout the agent code.
A small service can centralize the logic:
public interface IAgentCostCalculator
{
decimal Calculate(
long inputTokens,
long outputTokens,
int toolCalls);
}
Then the agent records usage:
var cost = costCalculator.Calculate(
usage.InputTokens,
usage.OutputTokens,
usage.ToolCalls);
This makes pricing changes easier to manage.
Common Mistakes
Measuring Only Model Costs
Tool and infrastructure costs can also matter.
Looking Only at Average Cost
A small number of expensive executions can have a large impact.
Ignoring Retries
Retries consume additional resources.
Allowing Unlimited Tool Calls
A faulty agent loop can generate unnecessary operations.
Sending Entire Tool Results to the Model
Large context increases processing and token usage.
Ignoring Context Growth
Conversation history can become a significant part of the input.
Testing Only Simple Prompts
Real production tasks are often more complex.
Tracking Cost Without Success Rate
A cheap agent that frequently fails may not be economically useful.
Best Practices
Measure Every Agent Run
Record enough information to reconstruct the execution cost.
Track Model Calls Individually
Do not treat an entire agent run as one model request.
Limit Retries
Set explicit retry limits and monitor retry rates.
Limit Tool Calls
Prevent accidental loops and unnecessary tool usage.
Control Context Size
Retrieve and send only relevant information.
Measure P95 and P99
Understand expensive outlier executions.
Test Failure Scenarios
A tool failure can change both latency and cost.
Set a Maximum Execution Budget
Give each run a practical upper limit.
Monitor Cost After Deployment
Production traffic can behave differently from test traffic.
Advantages of Measuring Cost Before Production
Makes unit economics visible.
Identifies expensive agent steps early.
Helps prevent runaway execution.
Improves capacity planning.
Makes model and architecture comparisons easier.
Provides data for production monitoring.
Disadvantages and Trade-Offs
Detailed telemetry adds implementation work.
Cost calculation can become more complicated when multiple providers are involved.
Testing realistic agent behavior requires representative scenarios.
Pricing changes can require updates to cost calculations.
Detailed telemetry must be designed carefully to avoid collecting sensitive information.
A Practical Pre-Production Checklist
[ ] Measure model calls per agent run
[ ] Track input tokens
[ ] Track output tokens
[ ] Measure tool calls
[ ] Measure retries
[ ] Track execution duration
[ ] Calculate cost per run
[ ] Calculate cost per successful task
[ ] Measure P95 and P99 cost
[ ] Test large-context scenarios
[ ] Test tool failures
[ ] Test retry behavior
[ ] Set maximum tool calls
[ ] Set maximum model calls
[ ] Set execution time limits
[ ] Add cost telemetry
[ ] Test realistic workloads
[ ] Estimate production traffic
[ ] Review sensitive telemetry fields
Conclusion
AI agents introduce a different cost model from traditional application requests because one user action can trigger multiple model calls, tool executions, retries, and supporting infrastructure operations.
The most useful approach is to measure the complete execution rather than focusing only on the model's individual request price.
Track model calls, tokens, tool usage, retries, execution time, successful outcomes, and cost per run. Then use realistic workloads to understand average and worst-case behavior before production.
A practical cost measurement flow looks like this:
User Request
|
v
Agent Run
|
+---- Model Calls
|
+---- Tool Calls
|
+---- Retries
|
+---- Context Growth
|
v
Usage Metrics
|
v
Cost Calculation
|
v
Cost Per Successful Task
|
v
Production Budget
The goal is not to make every agent execution as cheap as possible. The goal is to understand what each execution costs, where that cost comes from, and whether the cost is appropriate for the value the application provides.
That measurement should happen before the agent reaches production, not after an unexpected usage bill reveals that the original assumptions were wrong.
Join the conversation! Your thoughts help the community grow.