Building MCP Load Tests for Stateless Agent Workloads

Model Context Protocol (MCP) is becoming an important integration layer between AI applications and external tools, APIs, databases, and services.

As MCP workloads move from local development into shared environments, functional testing is no longer enough.

An MCP server may work perfectly for one client and still fail under concurrent agent workloads because of:

Connection limits
Request queuing
Thread-pool pressure
Database contention
External API throttling
Excessive allocations
Slow tool execution
Poor timeout handling
Incorrect state management

The challenge becomes even more interesting when MCP is exposed over HTTP using a stateless architecture.

A stateless MCP server should be able to process independent requests without requiring server-side conversational state to remain attached to a particular client connection.

That makes it a good candidate for horizontal scaling, but the scalability assumptions still need to be tested.

This article presents a practical approach to building MCP load tests for stateless agent workloads using .NET and standard load-testing concepts.

What Is a Stateless MCP Workload?

An MCP server exposes tools that an AI application can invoke.

A simplified architecture looks like this:

AI Application
      |
      v
MCP Client
      |
      v
HTTP
      |
      v
MCP Server
      |
      +---- Database
      |
      +---- REST API
      |
      +---- Internal Service

In a stateless design, the server should not depend on a particular HTTP connection to remember the client conversation.

Instead, each request contains enough information for the server to process that operation.

Conceptually:

Request A
   |
   v
Server Instance 1

Request B
   |
   v
Server Instance 2

Request C
   |
   v
Server Instance 3

This characteristic is useful for horizontal scaling.

However, statelessness does not automatically guarantee scalability.

A stateless server can still become CPU-bound, memory-bound, I/O-bound, or dependent on a bottleneck such as a database.

Why MCP Load Testing Is Different

A traditional API load test may send requests such as:

GET /api/orders/1001

An MCP workload is usually more tool-oriented.

For example:

Agent
  |
  +-- SearchOrders
  |
  +-- GetCustomer
  |
  +-- CreateTicket
  |
  +-- CalculateRefund

A realistic test therefore needs to model tool usage patterns, not simply generate random HTTP requests.

An agent might perform:

User Request
     |
     v
Search Customer
     |
     v
Read Orders
     |
     v
Analyze Result
     |
     v
Create Support Ticket

That sequence can create a very different workload from repeatedly calling one endpoint.

Define the Load-Test Objective

Before generating traffic, define what you want to measure.

Typical objectives include:

Maximum sustainable requests per second
Tool invocation throughput
p95 and p99 latency
Error rate
CPU utilization
Memory usage
GC behavior
Database pressure
External API latency
Scaling behavior

A useful test statement is:

Determine how the MCP server behaves
when N concurrent clients invoke
representative tools under a defined
request distribution.

Avoid vague goals such as:

Make the MCP server fast.

A measurable objective produces a more useful test.

Define the Workload Model

Start by listing the tools used by real workflows.

For example:

Tool	Workload Type	Expected Frequency
SearchCustomers	Read	High
GetCustomer	Read	High
GetOrder	Read	Medium
CreateTicket	Write	Low
CalculateRefund	CPU/business logic	Low
GenerateReport	Expensive	Very Low

This distribution is more realistic than assigning every tool the same probability.

For example:

SearchCustomers   40%
GetCustomer       25%
GetOrder          20%
CreateTicket      10%
GenerateReport     5%

The percentages should come from actual application behavior where possible.

Create a Representative MCP Request

A load test should use valid MCP messages rather than arbitrary JSON.

The exact MCP request structure depends on the MCP transport and SDK version being tested.

Conceptually, a tool invocation contains:

{
  "method": "tools/call",
  "params": {
    "name": "get_customer",
    "arguments": {
      "customerId": "CUST-1001"
    }
  }
}

The important part is that the benchmark client should exercise the same protocol path used by the production MCP client.

Do not replace the real MCP protocol with a synthetic endpoint and then claim that the results represent MCP performance.

Build a Dedicated Load-Test Project

Keep load tests separate from application tests.

For example:

Solution
 |
 +-- McpServer
 |
 +-- McpServer.Tests
 |
 +-- McpServer.IntegrationTests
 |
 +-- McpServer.LoadTests

The load-test project should contain:

Workload definitions
Client creation
Request generation
Concurrency configuration
Metrics collection
Result reporting

This makes performance testing repeatable.

Use HttpClient for HTTP-Level Tests

For a basic HTTP load test, HttpClient can generate requests.

For example:

using System.Net.Http.Json;

public sealed class McpLoadClient
{
    private readonly HttpClient _httpClient;

    public McpLoadClient(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }

    public async Task<HttpResponseMessage> CallToolAsync(
        string tool,
        object arguments,
        CancellationToken cancellationToken)
    {
        var request = new
        {
            method = "tools/call",
            @params = new
            {
                name = tool,
                arguments
            }
        };

        return await _httpClient.PostAsJsonAsync(
            "/mcp",
            request,
            cancellationToken);
    }
}

This is useful for validating the HTTP path, but a production-grade load test should also account for the actual MCP transport semantics being used.

Do Not Create a New HttpClient Per Request

Avoid:

using var client = new HttpClient();

inside every iteration.

Instead, reuse the client:

var client = new HttpClient
{
    BaseAddress =
        new Uri("https://localhost:5001")
};

or use IHttpClientFactory in a larger test harness.

Repeatedly creating clients can distort the results through unnecessary connection creation and resource usage.

The load generator itself should not become the bottleneck.

Generate Concurrent Requests

A simple concurrency test can use tasks:

var tasks = Enumerable
    .Range(0, concurrency)
    .Select(_ =>
        client.CallToolAsync(
            "get_customer",
            new
            {
                customerId = "CUST-1001"
            },
            CancellationToken.None));

var responses =
    await Task.WhenAll(tasks);

This creates concurrent requests, but it should not be confused with a complete load-testing framework.

A real load test needs controlled:

Ramp-up
Steady-state duration
Ramp-down
Concurrency
Request distribution
Failure handling

Ramp Up Gradually

Do not immediately start with thousands of concurrent clients.

Use stages:

10 clients
   |
   v
25 clients
   |
   v
50 clients
   |
   v
100 clients
   |
   v
250 clients
   |
   v
500 clients

At every stage, collect:

Latency
Throughput
Errors
CPU
Memory
GC
Database

This helps identify where the system begins to degrade.

Measure p50, p95, and p99

Average latency can hide serious problems.

Suppose a workload produces:

Most requests: 50 ms
A few requests: 5 seconds

The average may look acceptable while users experience significant tail latency.

Track:

Metric	Meaning
p50	Median request
p95	95% of requests are at or below this value
p99	99% of requests are at or below this value
Max	Slowest observed request

For agent workloads, tail latency can be particularly important because one slow tool invocation can delay an entire agent workflow.

Measure Tool-Level Latency

HTTP latency alone is not enough.

Consider:

HTTP Request
     |
     v
MCP Routing
     |
     v
Tool Execution
     |
     v
Database
     |
     v
Response

If the total request takes 500 ms, you need to know where that time went.

Add instrumentation around the tool:

var stopwatch =
    Stopwatch.StartNew();

try
{
    return await ExecuteToolAsync(
        arguments,
        cancellationToken);
}
finally
{
    stopwatch.Stop();

    metrics.RecordToolDuration(
        toolName,
        stopwatch.Elapsed);
}

The exact implementation can use OpenTelemetry or another metrics system.

Measure Error Rates

Track more than HTTP status codes.

An MCP request can fail because of:

Transport failure
Protocol error
Tool validation failure
Authorization failure
Database timeout
External API failure
Tool execution exception
Cancellation

Separate these categories.

For example:

Total requests: 100,000

Transport errors: 100
Validation errors: 200
Authorization failures: 50
Database timeouts: 75
Successful requests: 99,575

This is more actionable than simply reporting:

99.6% successful.

Test Statelessness Explicitly

Statelessness should be a testable property.

A useful experiment is:

Client
 |
 +---- Request 1 ----> Server A
 |
 +---- Request 2 ----> Server B
 |
 +---- Request 3 ----> Server C

The application should behave correctly without requiring all requests to reach the same server instance.

In a local environment, you can simulate this using multiple application instances.

For example:

Load Balancer
    |
    +---- Instance A
    |
    +---- Instance B
    |
    +---- Instance C

Then deliberately distribute requests across instances.

Test Horizontal Scaling

Run the same workload with different instance counts:

Instances	Concurrency	Throughput	p95	Error Rate
1	Fixed	Measure	Measure	Measure
2	Fixed	Measure	Measure	Measure
4	Fixed	Measure	Measure	Measure
8	Fixed	Measure	Measure	Measure

Do not invent expected scaling ratios.

A two-instance deployment will not necessarily provide exactly twice the throughput.

Shared resources can become bottlenecks.

For example:

Instance A --+
             |
Instance B --+--> Database
             |
Instance C --+

The database may become the limiting factor even when application CPU remains available.

Test Database Contention

MCP tools frequently interact with databases.

Suppose:

100 concurrent agents
       |
       v
MCP Server
       |
       v
SQL Server

The database may become the actual bottleneck.

Measure:

Connection pool usage
Query latency
Lock contention
CPU
Reads/writes
Timeout rate

A server benchmark without database measurements can produce misleading conclusions.

Test External Service Dependencies

Some MCP tools call third-party services.

For example:

MCP Server
    |
    v
Payment API

A load test against the MCP server can unintentionally overload the third-party service.

Use controlled mocks or sandbox services where appropriate.

For example:

Load Generator
      |
      v
MCP Server
      |
      v
Mock External API

This allows the MCP server itself to be measured without creating uncontrolled external traffic.

Test Rate Limits

External dependencies may enforce rate limits.

The load test should verify that the application handles them correctly.

For example:

HTTP 429
   |
   v
Retry Policy
   |
   v
Backoff
   |
   v
Retry

Do not create an aggressive retry loop.

A poorly configured retry policy can turn:

Service degradation

into:

Retry storm

Test Timeouts

Every external operation should have an intentional timeout.

For example:

using var timeout =
    CancellationTokenSource.CreateLinkedTokenSource(
        cancellationToken);

timeout.CancelAfter(
    TimeSpan.FromSeconds(10));

await ExecuteToolAsync(
    arguments,
    timeout.Token);

The timeout should reflect the actual tool's requirements.

Do not use an arbitrary timeout simply because it is easy to configure.

Load testing should verify that slow dependencies do not cause requests to remain active indefinitely.

Test Cancellation

Agent workloads may cancel tool calls.

For example:

User
 |
 v
Agent
 |
 v
Tool Request
 |
 X User cancels

The MCP server should respond appropriately.

A load test should include cancellation scenarios:

using var cts =
    new CancellationTokenSource();

var task =
    client.CallToolAsync(
        "generate_report",
        arguments,
        cts.Token);

cts.CancelAfter(
    TimeSpan.FromSeconds(2));

await Assert.ThrowsAnyAsync<
    OperationCanceledException>(
    () => task);

The exact exception behavior depends on the client implementation.

The important point is to verify that cancellation propagates rather than leaving expensive work running unnecessarily.

Test Long-Running Tools Separately

Not every tool has the same execution profile.

For example:

Fast tools:
GetCustomer
GetOrder

Medium tools:
SearchOrders
CreateTicket

Slow tools:
GenerateReport
ProcessLargeFile
RunAnalysis

Mixing all of them into one benchmark makes diagnosis difficult.

Run:

Fast-only workload
Slow-only workload
Mixed production workload

This provides a clearer picture of each bottleneck.

Test Payload Size

Request size can significantly affect performance.

Test multiple payload classes:

Small
Medium
Large
Very Large

For example:

{
  "customerId": "CUST-1001"
}

versus a large tool argument containing many records.

Measure:

Request size
Response size
Serialization time
Memory
Network transfer
Total latency

Large payloads can create pressure that is invisible in small functional tests.

Test Concurrent Tool Distribution

A realistic agent may not invoke one tool repeatedly.

Model the actual distribution.

For example:

var random = Random.Shared;

var tool = random.Next(100) switch
{
    < 40 => "search_customers",
    < 65 => "get_customer",
    < 85 => "get_order",
    < 95 => "create_ticket",
    _ => "generate_report"
};

For a serious benchmark, use a deterministic random seed or a predefined workload distribution so that runs can be compared.

The exact percentages should come from actual usage data when available.

Test Agent-Like Workflows

A tool-level benchmark is useful, but agent workflows are often sequential.

For example:

Search Customer
      |
      v
Get Orders
      |
      v
Analyze Orders
      |
      v
Create Ticket

A workflow benchmark can measure:

Total workflow latency
Tool count
Tool-level latency
Failure rate
Database calls
External calls

This gives a more realistic picture of user-visible performance.

Avoid Benchmarking the LLM Unless Necessary

If the goal is MCP server scalability, do not put an actual language model into the critical benchmark loop unless model latency is explicitly part of the research question.

An LLM introduces another variable:

Agent Workload
   |
   +-- Model latency
   |
   +-- MCP latency
   |
   +-- Tool latency
   |
   +-- Database latency

If the objective is MCP performance, generate deterministic MCP requests.

Then run a separate end-to-end experiment if model behavior itself needs to be measured.

Observe the Server During the Test

Load results without server telemetry are incomplete.

Monitor:

CPU
Memory
GC
Thread Pool
Requests
Connections
Database
Network
Exceptions

For .NET applications, OpenTelemetry can provide a useful observability foundation.

A simplified configuration might look like:

builder.Services
    .AddOpenTelemetry()
    .WithMetrics(metrics =>
    {
        metrics.AddAspNetCoreInstrumentation();
        metrics.AddRuntimeInstrumentation();
    });

The exact instrumentation packages and exporters depend on the environment.

Watch Thread Pool Behavior

High concurrency does not necessarily mean the application should create a matching number of threads.

For asynchronous .NET applications, monitor thread-pool behavior and blocking operations.

A tool that performs synchronous I/O inside an asynchronous request path can cause unexpected contention.

For example:

// Avoid blocking asynchronous work.
var result = task.Result;

Prefer:

var result =
    await task;

Load tests are useful for exposing these issues because blocking operations become more expensive as concurrency increases.

Test Memory Under Sustained Load

A short spike test may not reveal memory problems.

Run a sustained workload:

Ramp-up
   |
   v
Stable Load
   |
   v
30 minutes
   |
   v
60 minutes
   |
   v
Observe

Track:

Working Set
Managed Heap
GC Collections
Allocation Rate

A server that survives 30 seconds at high load may still develop memory pressure after sustained operation.

Do not publish a specific "safe duration" without evidence from the application's requirements.

Load Test Scenarios

A useful test suite can contain:

Scenario	Purpose
Baseline	Establish normal behavior
Ramp test	Find degradation point
Sustained load	Detect long-running issues
Spike test	Test sudden traffic
Mixed workload	Represent normal usage
Slow-tool test	Examine expensive operations
Failure test	Validate dependency failures
Cancellation test	Validate request cancellation
Scale-out test	Evaluate horizontal scaling
Recovery test	Evaluate behavior after overload

This is more useful than one large benchmark.

Example Load-Test Configuration

Keep workload configuration outside the source code where practical.

For example:

{
  "loadTest": {
    "durationSeconds": 300,
    "targetConcurrency": 100,
    "rampUpSeconds": 60,
    "coolDownSeconds": 30
  }
}

Then load it:

var configuration =
    await File.ReadAllTextAsync(
        "loadtest.json");

var settings =
    JsonSerializer.Deserialize<
        LoadTestSettings>(
            configuration);

This allows different environments to use different test profiles without changing the benchmark implementation.

Establish Acceptance Criteria

A benchmark is more useful when it has explicit success criteria.

For example:

Error rate < defined threshold
p95 latency < defined threshold
p99 latency < defined threshold
CPU < defined operational limit
No sustained memory growth
No database timeout spike

Do not copy these thresholds from another application.

Define them from your own service-level objectives and dependency characteristics.

Common Load-Testing Mistakes

Testing Only One Tool

This does not represent a multi-tool agent workload.

Using Random Requests Without a Distribution

Random traffic can produce unrealistic usage.

Ignoring Downstream Services

The MCP server may appear healthy while the database is overloaded.

Measuring Only Average Latency

Tail latency can be more important for agent workflows.

Running the Load Generator on the Same Small Machine

The client can become the bottleneck.

Using Real Production APIs During Development

This can create unintended external traffic.

Benchmarking an LLM When Testing MCP

Model latency can hide the actual MCP performance characteristics.

Ignoring Warm-Up

The first requests can include startup, JIT, connection establishment, or cache effects.

Publishing Results Without Environment Details

Performance results without runtime, hardware, configuration, dataset, and workload information are difficult to reproduce.

A Reproducible Benchmark Method

A good benchmark report should document:

.NET SDK
.NET runtime
MCP SDK version
Operating system
CPU
Memory
Container configuration
Server instance count
Database configuration
Tool distribution
Concurrency
Payload sizes
Test duration
Warm-up duration

Then record:

Requests/sec
p50
p95
p99
Error rate
CPU
Memory
GC
Database latency

This makes the experiment repeatable.

Recommended MCP Load-Test Architecture

A production-oriented setup can look like:

                 Load Generator
                       |
                       v
                 Load Balancer
                       |
          +------------+------------+
          |            |            |
          v            v            v
       MCP #1       MCP #2       MCP #3
          |            |            |
          +------------+------------+
                       |
              +--------+--------+
              |                 |
              v                 v
          Database        External APIs

Telemetry should flow separately:

MCP Instances
      |
      v
Metrics / Traces / Logs
      |
      v
Observability Platform

This separation helps distinguish application bottlenecks from infrastructure bottlenecks.

Best Practices

Test the real MCP transport.
Model actual tool distributions.
Use deterministic workloads where possible.
Measure p50, p95, and p99 latency.
Measure errors by category.
Test statelessness across multiple instances.
Measure downstream dependencies.
Test cancellation and timeouts.
Separate fast and expensive tools.
Run sustained-load tests.
Keep the load generator independent from the server.
Document the complete benchmark environment.
Avoid fabricated performance numbers.
Use production-like payloads and workflows.
Repeat tests after significant MCP SDK or infrastructure changes.

Frequently Asked Questions

What should an MCP load test measure?

At minimum, measure throughput, p50/p95/p99 latency, error rate, CPU, memory, and downstream dependency behavior.

For tool-heavy workloads, also capture tool-level latency.

Should I test every MCP tool?

Yes, if the goal is to understand production behavior.

Prioritize tools based on actual usage and risk, then create separate tests for expensive or high-volume operations.

Can I use ordinary HTTP load-testing tools?

Yes, provided they correctly implement the MCP transport and message format being tested.

A generic HTTP request benchmark that does not exercise the MCP protocol cannot reliably represent MCP performance.

Does stateless MCP automatically scale horizontally?

No.

Statelessness removes one category of scaling constraint, but databases, external services, CPU, memory, network bandwidth, connection pools, and rate limits can still become bottlenecks.

Should an actual AI model be included in the benchmark?

Only when model behavior and end-to-end agent performance are part of the objective.

For MCP server benchmarking, deterministic protocol requests usually provide cleaner measurements.

How do I know whether the MCP server or database is the bottleneck?

Correlate MCP request latency with server metrics and database telemetry.

If database latency increases while application CPU remains relatively stable, the database or its connection/query workload may be limiting throughput.

Conclusion

MCP load testing should be treated as a workload-engineering problem rather than simply sending large numbers of HTTP requests.

A useful test represents how agents actually use tools:

Agent Workflow
      |
      v
MCP Tool Calls
      |
      v
HTTP Transport
      |
      v
MCP Server
      |
      +---- Database
      |
      +---- External APIs

For stateless workloads, deliberately test whether requests can be distributed across multiple server instances without relying on connection-local state.

Then measure the complete system:

Concurrency
    +
Throughput
    +
Tail Latency
    +
Errors
    +
CPU
    +
Memory
    +
Database
    +
External Dependencies

The most important principle is:

Do not ask how many requests an MCP server can handle in isolation. Ask how the complete agent workload behaves as concurrency, tool complexity, payload size, and downstream dependencies increase.

That approach produces performance data that is useful for capacity planning, architecture decisions, and production readiness rather than a benchmark number that only applies to an artificial test.