Multi-tenant AI applications create a security problem that traditional web applications do not always face.
A normal SaaS application may process a request, query a database, return a response, and finish. An AI agent can be different. It may maintain conversational context, execute tools, create files, call external services, run commands, and continue working across multiple interactions.
That makes tenant isolation a critical part of the architecture.
Amazon Bedrock AgentCore Runtime addresses this by running each user session in a dedicated microVM with isolated CPU, memory, and filesystem resources. The session can preserve context across multiple invocations while remaining separated from other sessions. When the session terminates, the microVM is terminated and its memory is sanitized.
For teams considering AgentCore for multi-tenant AI systems, however, the architectural description is only the starting point.
The more useful engineering question is:
What happens to latency, startup behavior, resource consumption, and operational efficiency when many isolated AI sessions run concurrently?
This article presents a practical methodology for benchmarking those characteristics without inventing performance numbers that depend on a particular workload or AWS environment.
Understanding AgentCore Runtime Isolation
AgentCore Runtime is a managed hosting environment for AI agents and tools. It supports different agent frameworks and models and provides capabilities such as session management, authentication, isolation, and infrastructure management.
The basic multi-tenant architecture looks like this:
AgentCore Runtime
|
+----------------+----------------+
| | |
v v v
Session A Session B Session C
microVM microVM microVM
| | |
Tenant A Tenant B Tenant C
Each session receives its own execution environment.
AWS documents the isolation boundary as dedicated CPU, memory, and filesystem resources for each session. This is particularly relevant for agents because the runtime may contain intermediate reasoning state, files, tool results, and other session-specific information.
This differs from simply running multiple requests inside the same application process.
Why Multi-Tenant Agent Isolation Is Different
A traditional ASP.NET Core application might use:
One Process
|
+-- Tenant A Request
+-- Tenant B Request
+-- Tenant C Request
Application code is responsible for ensuring that tenant-specific state never crosses request boundaries.
With isolated AgentCore sessions:
Runtime
|
+-- MicroVM A --> Tenant A
|
+-- MicroVM B --> Tenant B
|
+-- MicroVM C --> Tenant C
The execution environment itself becomes part of the isolation model.
That does not eliminate application-level authorization.
AWS explicitly states that AgentCore does not enforce the mapping between users and session IDs. The client backend remains responsible for maintaining that relationship and managing session lifecycle policies such as the maximum number of sessions per user.
Therefore, isolation has two layers:
Infrastructure Isolation
+
Application Authorization
Both need to be tested.
What Should Be Benchmarked?
A useful isolation benchmark should not attempt to produce one universal "AgentCore performance" number.
Instead, measure several dimensions.
| Dimension | What to Measure |
|---|
| Startup | Time to first usable execution |
| Warm execution | Latency after session creation |
| Concurrency | Behavior with many active sessions |
| CPU | Resource consumption per session |
| Memory | Working-set behavior |
| Filesystem | Isolation and I/O behavior |
| Session lifecycle | Creation, idle, resume, termination |
| Cold starts | Additional latency after new microVM creation |
| Throughput | Completed operations over time |
| Errors | Failed or rejected operations |
| Tenant isolation | Cross-session access attempts |
The benchmark should clearly separate security correctness from performance.
A system can be fast and still have an unacceptable isolation failure.
Designing the Benchmark Environment
A controlled benchmark might use:
Test Controller
|
+-----------+-----------+
| | |
v v v
Tenant A Tenant B Tenant C
| | |
v v v
Session A Session B Session C
| | |
+-----------+-----------+
|
AgentCore Runtime
The controller should generate consistent workloads.
For example, each session could perform the same deterministic operation:
Create session
|
v
Initialize agent
|
v
Execute tool
|
v
Write temporary file
|
v
Read temporary file
|
v
Return result
This avoids allowing model variability to dominate the infrastructure benchmark.
If you want to evaluate actual agent workloads, add a second benchmark layer using the same model, prompts, tools, and configuration across all test runs.
Benchmark Cold and Warm Sessions Separately
One of the biggest mistakes in serverless benchmarking is treating every request as equivalent.
AgentCore sessions have lifecycle behavior.
AWS documents that the first invocation with a session ID creates the session and provisions its execution environment. Subsequent invocations using the same session ID reuse that environment while the session remains active.
Therefore, measure at least two scenarios.
Cold Session
Request
|
v
New Session
|
v
MicroVM Provisioning
|
v
Agent Initialization
|
v
Application Work
Warm Session
Existing Session
|
v
Existing MicroVM
|
v
Application Work
These numbers answer different questions.
Cold-session latency is important for sporadic users.
Warm-session latency is more relevant for active conversations.
Testing Session Affinity
AgentCore uses the session identifier to route subsequent requests to the same microVM. The session header depends on the protocol; for MCP, the documented header is Mcp-Session-Id, while HTTP and other protocols use the corresponding AgentCore runtime session header.
A benchmark should verify this behavior.
For example:
Tenant A
|
+--> Request 1
|
+--> Request 2
|
+--> Request 3
All three requests should use the same session identifier when they belong to the same conversation.
Then create:
Tenant B
|
+--> Request 1
with a different session ID.
The benchmark should verify that Tenant B does not receive Tenant A's session context.
Testing Cross-Tenant Filesystem Isolation
Filesystem isolation is particularly important for coding agents and agents that manipulate files.
Suppose Tenant A creates:
/workspace/tenant-a/report.txt
Tenant B should not be able to access it through its own session.
The test can deliberately attempt:
Tenant B
|
v
Read /workspace/tenant-a/report.txt
The expected result is failure.
Then verify that:
Tenant A
|
v
Read /workspace/tenant-a/report.txt
succeeds.
This is a security test rather than a performance test.
It should be executed repeatedly under concurrent load.
AWS documents that each session receives isolated filesystem resources and that one session cannot read or write another session's session storage when managed session storage is used.
Testing Persistent Session Storage
AgentCore also supports persistent filesystem configurations.
AWS currently documents managed session storage as a preview capability that persists across stop/resume cycles while remaining isolated per session. It also supports customer-managed Amazon S3 Files and Amazon EFS configurations, which can be shared across sessions and agents.
These are fundamentally different storage models.
| Storage | Isolation | Persistence | Shared? |
|---|
| Ephemeral session filesystem | Per session | Session lifecycle | No |
| Managed session storage | Per session | Stop/resume | No |
| S3 Files | Shared | Customer-managed | Yes |
| EFS | Shared | Customer-managed | Yes |
Do not use a shared filesystem when the requirement is strict per-tenant isolation unless the application explicitly implements tenant-level access controls.
Measuring Concurrent Sessions
The central benchmark should progressively increase the number of active sessions.
For example:
1 session
10 sessions
25 sessions
50 sessions
100 sessions
250 sessions
500 sessions
These are test levels, not claims about a production capacity limit.
At each level, measure:
Session creation latency
First response latency
Warm request latency
CPU usage
Memory usage
Error rate
Throughput
A results table can look like:
| Concurrent Sessions | Cold p50 | Warm p50 | p95 | Errors | CPU | Memory |
|---|
| 1 | Measure | Measure | Measure | Measure | Measure | Measure |
| 10 | Measure | Measure | Measure | Measure | Measure | Measure |
| 50 | Measure | Measure | Measure | Measure | Measure | Measure |
| 100 | Measure | Measure | Measure | Measure | Measure | Measure |
| 250 | Measure | Measure | Measure | Measure | Measure | Measure |
Populate these values from your own environment.
The purpose of the benchmark is to identify how the workload behaves as concurrency increases.
Measuring CPU and Memory Isolation
A useful test should also determine whether a noisy workload in one session affects another session.
Consider:
Session A
CPU-intensive operation
|
v
High CPU utilization
Session B
Simple request
|
v
Measure latency
If Session B's latency changes significantly as Session A becomes CPU-intensive, investigate whether the observed behavior comes from the runtime's isolation boundary, shared external dependencies, client-side bottlenecks, or the test environment.
The benchmark should therefore avoid assuming the cause before measuring it.
AgentCore documents CPU and memory isolation at the microVM session boundary.
Testing Memory-Heavy Agents
AI agents may hold substantial intermediate state.
A controlled workload can allocate memory inside a session:
data = bytearray(100 * 1024 * 1024)
The exact size should be selected according to the benchmark objective.
Run this workload in one session while monitoring another session.
Then repeat with multiple memory-heavy sessions.
The objective is to determine:
Whether the expected isolation behavior holds
How session resource consumption changes
Whether errors occur under pressure
Whether session startup or execution latency changes
Do not infer service-wide memory limits from a single test.
AWS's runtime isolation documentation describes each session as having isolated memory resources, but your workload still needs to be tested against the configuration and service limits applicable to your deployment.
Testing Long-Running Sessions
AgentCore Runtime supports sessions with execution lifetimes of up to eight hours. AWS documents separate lifecycle concepts including idle timeout and maximum lifetime.
A long-running benchmark should therefore test:
Create
|
v
Invoke
|
v
Idle
|
v
Resume
|
v
Invoke Again
Measure whether context and expected session state remain available after an idle period within the configured lifecycle.
Do not assume that a session remains alive indefinitely.
AWS documents that session termination can occur because of inactivity, maximum lifetime, or unhealthy state. A subsequent request using the same session ID can create a new execution environment after termination.
Testing Session Termination
A useful security test deliberately terminates a session and then attempts to access its previous state.
The expected lifecycle is:
Session A
|
v
Terminate
|
v
MicroVM destroyed
|
v
New session
|
v
No access to previous ephemeral state
AWS states that when a session terminates, its microVM is terminated and memory is sanitized.
For persistent storage, the expected behavior is different and must be tested according to the configured storage type.
Testing Tenant-to-Session Mapping
Infrastructure isolation does not automatically establish application identity.
Suppose your SaaS backend has:
User 101 -> Session A
User 202 -> Session B
The backend must maintain this relationship.
A malicious or buggy client should not be able to simply submit:
runtimeSessionId = Session-A
and claim to be User 202.
AWS explicitly states that AgentCore does not enforce the mapping between users and session IDs; this responsibility belongs to the client backend.
Therefore, include a test such as:
Authenticated User B
|
v
Attempts to invoke User A's session
|
v
Backend authorization
|
v
Request denied
This is one of the most important multi-tenant tests.
Testing Credentials Isolation
Agent sessions often need AWS permissions to access other resources.
That introduces another risk.
AWS notes that code running inside the microVM can access execution-role credentials through the metadata endpoint, so execution-role permissions must be carefully scoped.
A benchmark should therefore include credential-boundary tests.
For example:
Tenant A Agent
|
+--> Allowed resource
| |
| +--> Success
|
+--> Tenant B resource
|
+--> Denied
Do not assume microVM isolation alone provides application authorization.
The IAM policy must also enforce the intended access boundary.
Testing Noisy Neighbors
The noisy-neighbor test is particularly useful in multi-tenant systems.
Create two classes of sessions:
Normal Sessions
|
+--> Lightweight requests
Heavy Sessions
|
+--> CPU-intensive
+--> Memory-intensive
+--> File-intensive
Then measure normal-session behavior while increasing the number of heavy sessions.
Record:
Normal-session p50
Normal-session p95
Normal-session error rate
Heavy-session resource usage
If performance changes, investigate the entire dependency chain rather than immediately concluding that runtime isolation failed.
Potential shared resources include:
External APIs
Databases
Model endpoints
Network paths
Client infrastructure
Storage systems
AgentCore configuration
Benchmarking Shared Filesystems
Bring-your-own filesystem configurations behave differently from per-session storage.
AWS documents that S3 Files and EFS configurations can be shared across multiple sessions and agents. It also recommends handling concurrent access appropriately, for example through file-per-session naming patterns or advisory locking.
A concurrent test should therefore intentionally perform:
Session A --> write file
Session B --> write file
Session C --> read file
Then test conflicting writes:
Session A --> update same file
Session B --> update same file
The benchmark should determine whether your application-level locking strategy is sufficient.
The shared filesystem itself should not be treated as an isolation boundary.
Measuring Session Startup Overhead
For a multi-tenant application with infrequent requests, startup behavior may be more important than sustained throughput.
A simple measurement is:
T0 = Request sent
T1 = Session available
T2 = Agent starts processing
T3 = First response
Then calculate:
Session startup =
T1 - T0
and:
Time to first useful response =
T3 - T0
For warm sessions:
Warm response =
T3 - T0
The difference between cold and warm measurements can help determine whether session reuse is valuable for your workload.
Designing a Repeatable Test Harness
A benchmark controller can generate requests concurrently.
A simplified C# structure might look like:
var tasks = Enumerable
.Range(0, concurrency)
.Select(async tenantNumber =>
{
var sessionId = CreateSessionId(tenantNumber);
var start = Stopwatch.GetTimestamp();
await InvokeAgentAsync(
sessionId,
cancellationToken);
return Stopwatch.GetElapsedTime(start);
});
var results = await Task.WhenAll(tasks);
The important design decision is to keep the benchmark harness separate from the agent workload.
That lets you run the same workload against different runtime configurations.
For example:
Benchmark Harness
|
+--> Configuration A
|
+--> Configuration B
|
+--> Configuration C
Benchmarking With and Without Session Reuse
A particularly useful comparison is:
| Scenario | Session Strategy |
|---|
| A | New session for every request |
| B | Reuse session |
| C | Reuse session with moderate concurrency |
| D | Many independent sessions |
This shows whether session reuse changes application behavior.
For conversational agents, session reuse is generally fundamental because context is associated with the session. AWS documents that subsequent invocations using the same session ID can preserve context across interactions.
Do Not Benchmark Only the Runtime
AgentCore Runtime is only one component.
A realistic agent request may look like:
User
|
v
API
|
v
AgentCore Runtime
|
+--> Foundation Model
|
+--> AgentCore Gateway
|
+--> Database
|
+--> External APIs
|
+--> Storage
If response latency increases, the runtime may not be the bottleneck.
Therefore, capture timing at each layer where possible.
A useful trace might contain:
Request
|
+-- Authentication: 20 ms
|
+-- Runtime startup: 300 ms
|
+-- Agent processing: 500 ms
|
+-- Model: 900 ms
|
+-- Tool call: 200 ms
|
+-- Response: 50 ms
This prevents infrastructure benchmarks from becoming misleading application benchmarks.
Common Mistakes
Treating Isolation as Authorization
A dedicated microVM does not determine which user is allowed to use a session.
Reusing Session IDs Across Tenants
Each user or conversation should have an appropriately managed session identity.
Benchmarking Only Warm Sessions
Cold-session behavior can be important for low-frequency workloads.
Ignoring Shared Dependencies
Two isolated sessions can still compete for a shared database or external API.
Treating Shared Storage as Isolated
S3 Files and EFS can intentionally be shared across sessions.
Using Real LLM Variability for Every Test
Model latency and token generation can obscure infrastructure behavior.
Publishing Uncontrolled Benchmarks
A benchmark without hardware, workload, concurrency, configuration, and measurement details is difficult to reproduce.
Troubleshooting Unexpected Results
Warm requests are unexpectedly slow
Check:
Session affinity
Session lifecycle state
Model latency
External tool latency
Network configuration
AWS notes that without a consistent session ID, subsequent requests may be routed to a new microVM and incur additional startup latency.
Tenant data appears to be shared
Check:
Session-to-user mapping
Session IDs
Persistent storage configuration
Shared filesystem paths
Application-level caching
Database tenant filters
Do not assume the runtime is responsible for every application-level isolation decision.
CPU-intensive sessions affect other tenants
Measure external shared dependencies first and verify that sessions are actually independent.
Files persist when they should not
Check whether the application is using ephemeral storage, managed session storage, S3 Files, or EFS. These have different lifecycle and sharing semantics.
A resumed session behaves differently after deployment
AgentCore documents that existing sessions continue using the agent runtime version associated with the microVM created for that session until the session terminates and a new environment is created.
This should be considered during deployment and rollback planning.
Production Security Considerations
AWS's shared-responsibility model is important here.
AWS is responsible for infrastructure-level isolation and related managed infrastructure components.
Your team remains responsible for:
Agent code
Dependencies
IAM permissions
Session-to-user mapping
Network configuration
Container images
Input validation
Prompt-injection defenses
Commands executed by the agent
Secrets and credentials used by the application
This distinction should appear in the architecture documentation.
Do not describe "microVM isolation" as if it means the entire application is automatically secure.
Best Practices
Benchmark cold and warm sessions separately.
Test progressively increasing concurrent-session counts.
Measure p50, p95, and p99 latency.
Track CPU and memory behavior.
Test cross-session filesystem access explicitly.
Test session-to-user authorization independently.
Test credential boundaries with IAM.
Test noisy-neighbor scenarios.
Separate runtime measurements from model and downstream-service latency.
Test session termination and resumption behavior.
Document whether storage is isolated or shared.
Repeat tests across the actual configurations you intend to deploy.
Do not publish unsupported capacity claims from a single environment.
Treat security failures as release blockers regardless of performance results.
Conclusion
Multi-tenant AI systems need more than application-level tenant IDs.
Agents can maintain state, execute tools, manipulate files, and perform long-running operations. AgentCore Runtime addresses part of this problem by providing dedicated microVM-based session isolation, with isolated CPU, memory, and filesystem resources and lifecycle management around each session.
But infrastructure isolation is only one part of a secure multi-tenant architecture.
The application still needs to map users to sessions, enforce IAM permissions, protect shared resources, and control access to external systems. AWS explicitly assigns several of these responsibilities to the customer.
The best way to evaluate AgentCore Runtime for a production workload is therefore not to ask for one generic benchmark number.
Instead, build a controlled test suite covering:
Cold Start
+
Warm Session
+
Concurrent Sessions
+
Resource Consumption
+
Filesystem Isolation
+
Credential Isolation
+
Session Lifecycle
+
Shared Dependencies
Then compare those measurements against the actual requirements of your SaaS or enterprise AI workload.
The most valuable result is not a claim that one runtime is universally faster.
It is evidence showing how your workload behaves when multiple tenants run isolated agent sessions at the same time, where the real bottlenecks appear, and whether the security boundaries behave as designed.
Frequently Asked Questions
What does AgentCore Runtime isolate?
AgentCore Runtime provides each user session with a dedicated microVM with isolated CPU, memory, and filesystem resources.
Does AgentCore automatically map users to sessions?
No. AWS explicitly states that the client backend is responsible for maintaining the relationship between users and session IDs.
How long can an AgentCore Runtime session last?
AWS currently documents a maximum session lifetime of up to eight hours, with separate idle and maximum-lifetime controls.
Can sessions share persistent storage?
Yes, depending on the storage configuration. Managed session storage is isolated per session, while customer-managed S3 Files and EFS configurations can be shared across sessions and agents.
Should I benchmark AgentCore using a real LLM?
Use two layers. Start with deterministic workloads to isolate runtime behavior, then run a separate realistic benchmark using the actual model, tools, and prompts your application expects to use.
Does microVM isolation remove the need for IAM?
No. AWS recommends least-privilege IAM and explicitly identifies IAM access controls, resource policies, credentials, and session-to-user mappings as customer responsibilities.