Model Context Protocol (MCP) is becoming an important integration layer between AI applications and external tools, APIs, databases, and services.
As MCP workloads move from local development into shared environments, functional testing is no longer enough.
An MCP server may work perfectly for one client and still fail under concurrent agent workloads because of:
The challenge becomes even more interesting when MCP is exposed over HTTP using a stateless architecture.
A stateless MCP server should be able to process independent requests without requiring server-side conversational state to remain attached to a particular client connection.
That makes it a good candidate for horizontal scaling, but the scalability assumptions still need to be tested.
This article presents a practical approach to building MCP load tests for stateless agent workloads using .NET and standard load-testing concepts.
What Is a Stateless MCP Workload?
An MCP server exposes tools that an AI application can invoke.
A simplified architecture looks like this:
AI Application
|
v
MCP Client
|
v
HTTP
|
v
MCP Server
|
+---- Database
|
+---- REST API
|
+---- Internal Service
In a stateless design, the server should not depend on a particular HTTP connection to remember the client conversation.
Instead, each request contains enough information for the server to process that operation.
Conceptually:
Request A
|
v
Server Instance 1
Request B
|
v
Server Instance 2
Request C
|
v
Server Instance 3
This characteristic is useful for horizontal scaling.
However, statelessness does not automatically guarantee scalability.
A stateless server can still become CPU-bound, memory-bound, I/O-bound, or dependent on a bottleneck such as a database.
Why MCP Load Testing Is Different
A traditional API load test may send requests such as:
GET /api/orders/1001
An MCP workload is usually more tool-oriented.
For example:
Agent
|
+-- SearchOrders
|
+-- GetCustomer
|
+-- CreateTicket
|
+-- CalculateRefund
A realistic test therefore needs to model tool usage patterns, not simply generate random HTTP requests.
An agent might perform:
User Request
|
v
Search Customer
|
v
Read Orders
|
v
Analyze Result
|
v
Create Support Ticket
That sequence can create a very different workload from repeatedly calling one endpoint.
Define the Load-Test Objective
Before generating traffic, define what you want to measure.
Typical objectives include:
A useful test statement is:
Determine how the MCP server behaves
when N concurrent clients invoke
representative tools under a defined
request distribution.
Avoid vague goals such as:
Make the MCP server fast.
A measurable objective produces a more useful test.
Define the Workload Model
Start by listing the tools used by real workflows.
For example:
| Tool | Workload Type | Expected Frequency |
|---|
| SearchCustomers | Read | High |
| GetCustomer | Read | High |
| GetOrder | Read | Medium |
| CreateTicket | Write | Low |
| CalculateRefund | CPU/business logic | Low |
| GenerateReport | Expensive | Very Low |
This distribution is more realistic than assigning every tool the same probability.
For example:
SearchCustomers 40%
GetCustomer 25%
GetOrder 20%
CreateTicket 10%
GenerateReport 5%
The percentages should come from actual application behavior where possible.
Create a Representative MCP Request
A load test should use valid MCP messages rather than arbitrary JSON.
The exact MCP request structure depends on the MCP transport and SDK version being tested.
Conceptually, a tool invocation contains:
{
"method": "tools/call",
"params": {
"name": "get_customer",
"arguments": {
"customerId": "CUST-1001"
}
}
}
The important part is that the benchmark client should exercise the same protocol path used by the production MCP client.
Do not replace the real MCP protocol with a synthetic endpoint and then claim that the results represent MCP performance.
Build a Dedicated Load-Test Project
Keep load tests separate from application tests.
For example:
Solution
|
+-- McpServer
|
+-- McpServer.Tests
|
+-- McpServer.IntegrationTests
|
+-- McpServer.LoadTests
The load-test project should contain:
Workload definitions
Client creation
Request generation
Concurrency configuration
Metrics collection
Result reporting
This makes performance testing repeatable.
Use HttpClient for HTTP-Level Tests
For a basic HTTP load test, HttpClient can generate requests.
For example:
using System.Net.Http.Json;
public sealed class McpLoadClient
{
private readonly HttpClient _httpClient;
public McpLoadClient(HttpClient httpClient)
{
_httpClient = httpClient;
}
public async Task<HttpResponseMessage> CallToolAsync(
string tool,
object arguments,
CancellationToken cancellationToken)
{
var request = new
{
method = "tools/call",
@params = new
{
name = tool,
arguments
}
};
return await _httpClient.PostAsJsonAsync(
"/mcp",
request,
cancellationToken);
}
}
This is useful for validating the HTTP path, but a production-grade load test should also account for the actual MCP transport semantics being used.
Do Not Create a New HttpClient Per Request
Avoid:
using var client = new HttpClient();
inside every iteration.
Instead, reuse the client:
var client = new HttpClient
{
BaseAddress =
new Uri("https://localhost:5001")
};
or use IHttpClientFactory in a larger test harness.
Repeatedly creating clients can distort the results through unnecessary connection creation and resource usage.
The load generator itself should not become the bottleneck.
Generate Concurrent Requests
A simple concurrency test can use tasks:
var tasks = Enumerable
.Range(0, concurrency)
.Select(_ =>
client.CallToolAsync(
"get_customer",
new
{
customerId = "CUST-1001"
},
CancellationToken.None));
var responses =
await Task.WhenAll(tasks);
This creates concurrent requests, but it should not be confused with a complete load-testing framework.
A real load test needs controlled:
Ramp-up
Steady-state duration
Ramp-down
Concurrency
Request distribution
Failure handling
Ramp Up Gradually
Do not immediately start with thousands of concurrent clients.
Use stages:
10 clients
|
v
25 clients
|
v
50 clients
|
v
100 clients
|
v
250 clients
|
v
500 clients
At every stage, collect:
Latency
Throughput
Errors
CPU
Memory
GC
Database
This helps identify where the system begins to degrade.
Measure p50, p95, and p99
Average latency can hide serious problems.
Suppose a workload produces:
Most requests: 50 ms
A few requests: 5 seconds
The average may look acceptable while users experience significant tail latency.
Track:
| Metric | Meaning |
|---|
| p50 | Median request |
| p95 | 95% of requests are at or below this value |
| p99 | 99% of requests are at or below this value |
| Max | Slowest observed request |
For agent workloads, tail latency can be particularly important because one slow tool invocation can delay an entire agent workflow.
Measure Tool-Level Latency
HTTP latency alone is not enough.
Consider:
HTTP Request
|
v
MCP Routing
|
v
Tool Execution
|
v
Database
|
v
Response
If the total request takes 500 ms, you need to know where that time went.
Add instrumentation around the tool:
var stopwatch =
Stopwatch.StartNew();
try
{
return await ExecuteToolAsync(
arguments,
cancellationToken);
}
finally
{
stopwatch.Stop();
metrics.RecordToolDuration(
toolName,
stopwatch.Elapsed);
}
The exact implementation can use OpenTelemetry or another metrics system.
Measure Error Rates
Track more than HTTP status codes.
An MCP request can fail because of:
Transport failure
Protocol error
Tool validation failure
Authorization failure
Database timeout
External API failure
Tool execution exception
Cancellation
Separate these categories.
For example:
Total requests: 100,000
Transport errors: 100
Validation errors: 200
Authorization failures: 50
Database timeouts: 75
Successful requests: 99,575
This is more actionable than simply reporting:
99.6% successful.
Test Statelessness Explicitly
Statelessness should be a testable property.
A useful experiment is:
Client
|
+---- Request 1 ----> Server A
|
+---- Request 2 ----> Server B
|
+---- Request 3 ----> Server C
The application should behave correctly without requiring all requests to reach the same server instance.
In a local environment, you can simulate this using multiple application instances.
For example:
Load Balancer
|
+---- Instance A
|
+---- Instance B
|
+---- Instance C
Then deliberately distribute requests across instances.
Test Horizontal Scaling
Run the same workload with different instance counts:
| Instances | Concurrency | Throughput | p95 | Error Rate |
|---|
| 1 | Fixed | Measure | Measure | Measure |
| 2 | Fixed | Measure | Measure | Measure |
| 4 | Fixed | Measure | Measure | Measure |
| 8 | Fixed | Measure | Measure | Measure |
Do not invent expected scaling ratios.
A two-instance deployment will not necessarily provide exactly twice the throughput.
Shared resources can become bottlenecks.
For example:
Instance A --+
|
Instance B --+--> Database
|
Instance C --+
The database may become the limiting factor even when application CPU remains available.
Test Database Contention
MCP tools frequently interact with databases.
Suppose:
100 concurrent agents
|
v
MCP Server
|
v
SQL Server
The database may become the actual bottleneck.
Measure:
Connection pool usage
Query latency
Lock contention
CPU
Reads/writes
Timeout rate
A server benchmark without database measurements can produce misleading conclusions.
Test External Service Dependencies
Some MCP tools call third-party services.
For example:
MCP Server
|
v
Payment API
A load test against the MCP server can unintentionally overload the third-party service.
Use controlled mocks or sandbox services where appropriate.
For example:
Load Generator
|
v
MCP Server
|
v
Mock External API
This allows the MCP server itself to be measured without creating uncontrolled external traffic.
Test Rate Limits
External dependencies may enforce rate limits.
The load test should verify that the application handles them correctly.
For example:
HTTP 429
|
v
Retry Policy
|
v
Backoff
|
v
Retry
Do not create an aggressive retry loop.
A poorly configured retry policy can turn:
Service degradation
into:
Retry storm
Test Timeouts
Every external operation should have an intentional timeout.
For example:
using var timeout =
CancellationTokenSource.CreateLinkedTokenSource(
cancellationToken);
timeout.CancelAfter(
TimeSpan.FromSeconds(10));
await ExecuteToolAsync(
arguments,
timeout.Token);
The timeout should reflect the actual tool's requirements.
Do not use an arbitrary timeout simply because it is easy to configure.
Load testing should verify that slow dependencies do not cause requests to remain active indefinitely.
Test Cancellation
Agent workloads may cancel tool calls.
For example:
User
|
v
Agent
|
v
Tool Request
|
X User cancels
The MCP server should respond appropriately.
A load test should include cancellation scenarios:
using var cts =
new CancellationTokenSource();
var task =
client.CallToolAsync(
"generate_report",
arguments,
cts.Token);
cts.CancelAfter(
TimeSpan.FromSeconds(2));
await Assert.ThrowsAnyAsync<
OperationCanceledException>(
() => task);
The exact exception behavior depends on the client implementation.
The important point is to verify that cancellation propagates rather than leaving expensive work running unnecessarily.
Test Long-Running Tools Separately
Not every tool has the same execution profile.
For example:
Fast tools:
GetCustomer
GetOrder
Medium tools:
SearchOrders
CreateTicket
Slow tools:
GenerateReport
ProcessLargeFile
RunAnalysis
Mixing all of them into one benchmark makes diagnosis difficult.
Run:
Fast-only workload
Slow-only workload
Mixed production workload
This provides a clearer picture of each bottleneck.
Test Payload Size
Request size can significantly affect performance.
Test multiple payload classes:
Small
Medium
Large
Very Large
For example:
{
"customerId": "CUST-1001"
}
versus a large tool argument containing many records.
Measure:
Request size
Response size
Serialization time
Memory
Network transfer
Total latency
Large payloads can create pressure that is invisible in small functional tests.
Test Concurrent Tool Distribution
A realistic agent may not invoke one tool repeatedly.
Model the actual distribution.
For example:
var random = Random.Shared;
var tool = random.Next(100) switch
{
< 40 => "search_customers",
< 65 => "get_customer",
< 85 => "get_order",
< 95 => "create_ticket",
_ => "generate_report"
};
For a serious benchmark, use a deterministic random seed or a predefined workload distribution so that runs can be compared.
The exact percentages should come from actual usage data when available.
Test Agent-Like Workflows
A tool-level benchmark is useful, but agent workflows are often sequential.
For example:
Search Customer
|
v
Get Orders
|
v
Analyze Orders
|
v
Create Ticket
A workflow benchmark can measure:
Total workflow latency
Tool count
Tool-level latency
Failure rate
Database calls
External calls
This gives a more realistic picture of user-visible performance.
Avoid Benchmarking the LLM Unless Necessary
If the goal is MCP server scalability, do not put an actual language model into the critical benchmark loop unless model latency is explicitly part of the research question.
An LLM introduces another variable:
Agent Workload
|
+-- Model latency
|
+-- MCP latency
|
+-- Tool latency
|
+-- Database latency
If the objective is MCP performance, generate deterministic MCP requests.
Then run a separate end-to-end experiment if model behavior itself needs to be measured.
Observe the Server During the Test
Load results without server telemetry are incomplete.
Monitor:
CPU
Memory
GC
Thread Pool
Requests
Connections
Database
Network
Exceptions
For .NET applications, OpenTelemetry can provide a useful observability foundation.
A simplified configuration might look like:
builder.Services
.AddOpenTelemetry()
.WithMetrics(metrics =>
{
metrics.AddAspNetCoreInstrumentation();
metrics.AddRuntimeInstrumentation();
});
The exact instrumentation packages and exporters depend on the environment.
Watch Thread Pool Behavior
High concurrency does not necessarily mean the application should create a matching number of threads.
For asynchronous .NET applications, monitor thread-pool behavior and blocking operations.
A tool that performs synchronous I/O inside an asynchronous request path can cause unexpected contention.
For example:
// Avoid blocking asynchronous work.
var result = task.Result;
Prefer:
var result =
await task;
Load tests are useful for exposing these issues because blocking operations become more expensive as concurrency increases.
Test Memory Under Sustained Load
A short spike test may not reveal memory problems.
Run a sustained workload:
Ramp-up
|
v
Stable Load
|
v
30 minutes
|
v
60 minutes
|
v
Observe
Track:
Working Set
Managed Heap
GC Collections
Allocation Rate
A server that survives 30 seconds at high load may still develop memory pressure after sustained operation.
Do not publish a specific "safe duration" without evidence from the application's requirements.
Load Test Scenarios
A useful test suite can contain:
| Scenario | Purpose |
|---|
| Baseline | Establish normal behavior |
| Ramp test | Find degradation point |
| Sustained load | Detect long-running issues |
| Spike test | Test sudden traffic |
| Mixed workload | Represent normal usage |
| Slow-tool test | Examine expensive operations |
| Failure test | Validate dependency failures |
| Cancellation test | Validate request cancellation |
| Scale-out test | Evaluate horizontal scaling |
| Recovery test | Evaluate behavior after overload |
This is more useful than one large benchmark.
Example Load-Test Configuration
Keep workload configuration outside the source code where practical.
For example:
{
"loadTest": {
"durationSeconds": 300,
"targetConcurrency": 100,
"rampUpSeconds": 60,
"coolDownSeconds": 30
}
}
Then load it:
var configuration =
await File.ReadAllTextAsync(
"loadtest.json");
var settings =
JsonSerializer.Deserialize<
LoadTestSettings>(
configuration);
This allows different environments to use different test profiles without changing the benchmark implementation.
Establish Acceptance Criteria
A benchmark is more useful when it has explicit success criteria.
For example:
Error rate < defined threshold
p95 latency < defined threshold
p99 latency < defined threshold
CPU < defined operational limit
No sustained memory growth
No database timeout spike
Do not copy these thresholds from another application.
Define them from your own service-level objectives and dependency characteristics.
Common Load-Testing Mistakes
Testing Only One Tool
This does not represent a multi-tool agent workload.
Using Random Requests Without a Distribution
Random traffic can produce unrealistic usage.
Ignoring Downstream Services
The MCP server may appear healthy while the database is overloaded.
Measuring Only Average Latency
Tail latency can be more important for agent workflows.
Running the Load Generator on the Same Small Machine
The client can become the bottleneck.
Using Real Production APIs During Development
This can create unintended external traffic.
Benchmarking an LLM When Testing MCP
Model latency can hide the actual MCP performance characteristics.
Ignoring Warm-Up
The first requests can include startup, JIT, connection establishment, or cache effects.
Publishing Results Without Environment Details
Performance results without runtime, hardware, configuration, dataset, and workload information are difficult to reproduce.
A Reproducible Benchmark Method
A good benchmark report should document:
.NET SDK
.NET runtime
MCP SDK version
Operating system
CPU
Memory
Container configuration
Server instance count
Database configuration
Tool distribution
Concurrency
Payload sizes
Test duration
Warm-up duration
Then record:
Requests/sec
p50
p95
p99
Error rate
CPU
Memory
GC
Database latency
This makes the experiment repeatable.
Recommended MCP Load-Test Architecture
A production-oriented setup can look like:
Load Generator
|
v
Load Balancer
|
+------------+------------+
| | |
v v v
MCP #1 MCP #2 MCP #3
| | |
+------------+------------+
|
+--------+--------+
| |
v v
Database External APIs
Telemetry should flow separately:
MCP Instances
|
v
Metrics / Traces / Logs
|
v
Observability Platform
This separation helps distinguish application bottlenecks from infrastructure bottlenecks.
Best Practices
Test the real MCP transport.
Model actual tool distributions.
Use deterministic workloads where possible.
Measure p50, p95, and p99 latency.
Measure errors by category.
Test statelessness across multiple instances.
Measure downstream dependencies.
Test cancellation and timeouts.
Separate fast and expensive tools.
Run sustained-load tests.
Keep the load generator independent from the server.
Document the complete benchmark environment.
Avoid fabricated performance numbers.
Use production-like payloads and workflows.
Repeat tests after significant MCP SDK or infrastructure changes.
Frequently Asked Questions
What should an MCP load test measure?
At minimum, measure throughput, p50/p95/p99 latency, error rate, CPU, memory, and downstream dependency behavior.
For tool-heavy workloads, also capture tool-level latency.
Should I test every MCP tool?
Yes, if the goal is to understand production behavior.
Prioritize tools based on actual usage and risk, then create separate tests for expensive or high-volume operations.
Can I use ordinary HTTP load-testing tools?
Yes, provided they correctly implement the MCP transport and message format being tested.
A generic HTTP request benchmark that does not exercise the MCP protocol cannot reliably represent MCP performance.
Does stateless MCP automatically scale horizontally?
No.
Statelessness removes one category of scaling constraint, but databases, external services, CPU, memory, network bandwidth, connection pools, and rate limits can still become bottlenecks.
Should an actual AI model be included in the benchmark?
Only when model behavior and end-to-end agent performance are part of the objective.
For MCP server benchmarking, deterministic protocol requests usually provide cleaner measurements.
How do I know whether the MCP server or database is the bottleneck?
Correlate MCP request latency with server metrics and database telemetry.
If database latency increases while application CPU remains relatively stable, the database or its connection/query workload may be limiting throughput.
Conclusion
MCP load testing should be treated as a workload-engineering problem rather than simply sending large numbers of HTTP requests.
A useful test represents how agents actually use tools:
Agent Workflow
|
v
MCP Tool Calls
|
v
HTTP Transport
|
v
MCP Server
|
+---- Database
|
+---- External APIs
For stateless workloads, deliberately test whether requests can be distributed across multiple server instances without relying on connection-local state.
Then measure the complete system:
Concurrency
+
Throughput
+
Tail Latency
+
Errors
+
CPU
+
Memory
+
Database
+
External Dependencies
The most important principle is:
Do not ask how many requests an MCP server can handle in isolation. Ask how the complete agent workload behaves as concurrency, tool complexity, payload size, and downstream dependencies increase.
That approach produces performance data that is useful for capacity planning, architecture decisions, and production readiness rather than a benchmark number that only applies to an artificial test.