Running Untrusted Code in AWS Without Giving It Full Container Access

Aarav Patel

Running code that comes from users, customers, plugins, or automated systems is one of the harder problems in cloud application security. The code may need CPU, memory, networking, and temporary storage, but giving it access to the same container environment as the main application can create a much larger security risk.

This is especially important for applications that execute user-submitted scripts, AI-generated code, build steps, data-processing jobs, or custom plugins.

A safer approach is to isolate the untrusted workload and give it only the resources and permissions it actually needs. AWS provides several building blocks for this, including containers, IAM, networking controls, and sandboxing technologies.

The goal is not simply to run the code in another container. The goal is to make sure that a compromised workload has as little opportunity as possible to affect the rest of the system.

Why Running Untrusted Code Is Different

Normal application code is controlled by the development team. Untrusted code is not.

For example, imagine an application that allows users to submit Python or JavaScript code for execution. A normal request might look like this:

User
  |
  v
API
  |
  v
Code Execution Service
  |
  v
Sandbox

The execution service receives the code and starts an isolated environment.

The problem begins if the execution environment has unnecessary access to:

Application secrets
IAM credentials
Internal services
Production databases
Private network resources
Host-level resources
Container metadata
Shared storage

Even if the application itself is secure, an attacker may use the code execution feature as a path toward other resources.

For example, code such as this should never have access to your application's environment variables:

import os

for key, value in os.environ.items():
    print(key, value)

If the execution environment contains database credentials or API keys, those values can potentially become exposed.

Use a Separate Execution Boundary

The first design decision should be to separate the execution environment from the application that receives the request.

A typical architecture can look like this:

                    +----------------+
                    |     Client     |
                    +-------+--------+
                            |
                            v
                    +----------------+
                    |       API      |
                    +-------+--------+
                            |
                     Queue / Job
                            |
                            v
                  +-------------------+
                  | Execution Service |
                  +---------+---------+
                            |
                            v
                  +-------------------+
                  | Isolated Sandbox  |
                  |                   |
                  | Untrusted Code    |
                  +---------+---------+
                            |
                       Limited access
                            |
                +-----------+-----------+
                | Temporary Storage    |
                | Specific API Access  |
                +----------------------+

The important part is the boundary between the application and the code being executed.

The sandbox should not be treated as an extension of the main application container.

Why a Normal Container Is Not Always Enough

Containers provide process and filesystem isolation, but they should not automatically be treated as a complete security boundary for hostile workloads.

A container running arbitrary code may still have access to resources that the application does not intend to expose.

For example, a poorly configured workload might receive:

Environment variables
IAM credentials
Network access
Shared volumes
Service endpoints
Container metadata

The security model should therefore assume that the code running inside the sandbox may attempt to escape its intended boundaries.

A better approach is defense in depth.

This means combining:

Workload isolation
Least-privilege IAM
Network restrictions
Short execution timeouts
CPU and memory limits
Ephemeral storage
Minimal runtime images
Logging and monitoring
Input and output controls

No single control should be responsible for the entire security model.

Isolate the Workload With AWS Nitro-Based Sandboxing

For workloads that require a stronger isolation boundary, AWS Nitro-based sandboxing can be used to isolate untrusted workloads from the surrounding application environment.

The basic idea is that the untrusted workload runs inside a strongly isolated execution environment instead of sharing the same application runtime.

Conceptually:

Main Application
       |
       | Job
       v
Execution Manager
       |
       v
+----------------------+
| Nitro-based Sandbox  |
|                      |
| Untrusted Workload   |
| Temporary Files      |
| Limited Resources    |
+----------------------+

The exact implementation depends on the AWS service and sandboxing technology being used, but the architectural principle remains the same: reduce the amount of trust between the host application and the code being executed.

This is particularly useful for workloads where the code itself cannot be trusted.

Give the Sandbox a Separate IAM Role

IAM permissions should be designed around the workload rather than the application.

For example, suppose a code execution job only needs to read a specific object from S3.

Do not give the execution environment:

AmazonS3FullAccess

Instead, create a dedicated IAM role with only the required permission.

A simplified policy might look like:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject"
      ],
      "Resource": "arn:aws:s3:::my-execution-inputs/jobs/*"
    }
  ]
}

The principle is simple:

Sandbox permissions != Application permissions

The application may need access to databases, queues, configuration services, and other AWS resources.

The sandbox usually does not.

Keeping those permissions separate limits the damage if the untrusted workload is compromised.

Avoid Passing Application Secrets Into the Sandbox

One common mistake is to pass secrets through environment variables because it is convenient.

For example:

DATABASE_URL=...
API_KEY=...
JWT_SECRET=...
AWS_ACCESS_KEY_ID=...

A sandbox running arbitrary code should not receive these values unless they are explicitly required.

Instead, the application should communicate with the sandbox using controlled inputs and outputs.

For example:

Application
    |
    | Input JSON
    v
Sandbox
    |
    | Result JSON
    v
Application

The sandbox should receive only the data required to perform the job.

This makes the security boundary much easier to reason about.

Restrict Network Access

Network access is another important part of the isolation model.

An untrusted workload generally should not have unrestricted outbound access.

For example, if the code execution job only needs to process a local input file, it may not need Internet access at all.

A restrictive model could look like this:

Sandbox
   |
   +-- Allowed: Object storage
   |
   +-- Allowed: Specific internal API
   |
   +-- Blocked: Database
   |
   +-- Blocked: Metadata services
   |
   +-- Blocked: Internal administration endpoints
   |
   +-- Blocked: Unrestricted Internet

This is particularly important because a compromised process can use network connectivity to retrieve additional payloads or attempt to reach internal services.

Network isolation should therefore be designed as part of the execution architecture rather than added later.

Put Resource Limits on Every Job

Untrusted code can create another type of security problem: resource exhaustion.

A malicious or poorly written program could consume CPU indefinitely:

while True:
    pass

It could also allocate excessive memory:

data = []

while True:
    data.append("x" * 1024 * 1024)

The sandbox should therefore enforce limits such as:

Maximum execution time
Maximum CPU
Maximum memory
Maximum temporary storage
Maximum output size
Maximum number of concurrent jobs

For example:

Job timeout:       30 seconds
Memory limit:      512 MB
CPU limit:         1 vCPU
Storage limit:     100 MB
Output limit:      10 MB

These values should be based on the actual workload rather than copied directly into production.

Make the Execution Environment Ephemeral

A sandbox should ideally be disposable.

Instead of maintaining a long-lived environment:

Sandbox A
  Job 1
  Job 2
  Job 3
  Job 4

prefer a model where the environment is created for a workload and discarded afterward:

Job 1 -> Sandbox -> Destroy
Job 2 -> Sandbox -> Destroy
Job 3 -> Sandbox -> Destroy

This reduces the chance that data from one execution remains available to another execution.

Ephemeral environments are particularly useful when different users submit unrelated workloads.

Validate the Input Before Execution

Isolation does not remove the need for input validation.

The application should validate:

Maximum source-code size
Supported runtime
Input file size
Command parameters
Job timeout
Resource requirements
Output size

For example, an API should not allow a client to request:

{
  "timeout": 86400,
  "memory": "128GB",
  "cpu": 64
}

unless the system is intentionally designed to support those values.

The execution service should apply its own limits rather than trusting the client.

Capture Logs Without Exposing Sensitive Data

Execution logs are useful for debugging and security monitoring.

However, logging everything produced by untrusted code can create another security problem.

For example, user code might print:

print(os.environ)

If those logs are stored permanently, sensitive information could be exposed through the logging system.

A better logging design separates:

Execution metadata
    |
    +-- Job ID
    +-- Start time
    +-- Duration
    +-- Exit status
    +-- Resource usage

Application output
    |
    +-- Controlled
    +-- Size limited
    +-- Sanitized where appropriate

The execution platform should also make it possible to identify which user or application request created each job.

Common Mistakes

Reusing the Application Container

Running untrusted code directly inside the API container creates unnecessary risk.

The application container should remain responsible for application logic, not arbitrary user code.

Giving the Sandbox Broad IAM Permissions

A role such as AdministratorAccess defeats the purpose of isolation.

The execution role should contain only the permissions required by the workload.

Leaving Network Access Open

A sandbox with unrestricted outbound and internal connectivity provides an attacker with more opportunities to move beyond the execution environment.

Keeping Sandboxes Alive

Long-lived execution environments can retain files, processes, and state that should not survive between jobs.

Trusting Client-Provided Resource Limits

Resource limits should be enforced by the execution service.

A client should not be able to request unlimited CPU, memory, storage, or execution time.

A Practical Security Checklist

Before moving an untrusted execution system into production, verify the following:

Area	Recommended approach
Isolation	Use a dedicated sandbox boundary
IAM	Separate execution role
Secrets	Do not expose application secrets
Network	Allow only required destinations
Runtime	Use a minimal environment
CPU	Enforce a fixed or bounded limit
Memory	Enforce a bounded limit
Storage	Use temporary storage
Timeout	Kill jobs that exceed the limit
Output	Limit and control output size
Logging	Capture execution metadata safely
Lifecycle	Prefer ephemeral execution
Monitoring	Track failures and unusual resource usage

When This Architecture Makes Sense

Strong workload isolation is useful when an application executes code that it does not fully control.

Examples include:

Online code runners
AI-generated code execution
Plugin systems
Customer-defined scripts
Data transformation jobs
Build systems
Automated testing platforms
Document or file processing
Multi-tenant developer platforms

For completely trusted application code, this level of isolation may add unnecessary operational complexity.

The security boundary should match the trust boundary.

Final Thoughts

Running untrusted code safely in AWS requires more than putting the code inside a container.

The important design decision is to treat the workload as potentially hostile and limit what it can access.

A strong architecture separates the execution environment from the main application, uses least-privilege IAM, restricts network connectivity, applies resource limits, avoids exposing secrets, and preferably uses ephemeral execution environments.

For workloads that require stronger isolation, Nitro-based sandboxing can provide an additional boundary between untrusted code and the surrounding infrastructure.

The result is a system where even if an individual execution job behaves maliciously, the potential impact is limited to the resources that the sandbox was intentionally allowed to use.

AWSuntrusted codesandboxingAWS NitroIAM least privilegecontainer securityworkload isolation

Comments

Join the conversation! Your thoughts help the community grow.

Running Untrusted Code in AWS Without Giving It Full Container Access

Aarav Patel

4h4.7k00100Article

This is especially important for applications that execute user-submitted scripts, AI-generated code, build steps, data-processing jobs, or custom plugins.

The goal is not simply to run the code in another container. The goal is to make sure that a compromised workload has as little opportunity as possible to affect the rest of the system.

Why Running Untrusted Code Is Different

Normal application code is controlled by the development team. Untrusted code is not.

For example, imagine an application that allows users to submit Python or JavaScript code for execution. A normal request might look like this:

User
  |
  v
API
  |
  v
Code Execution Service
  |
  v
Sandbox

The execution service receives the code and starts an isolated environment.

The problem begins if the execution environment has unnecessary access to:

Application secrets
IAM credentials
Internal services
Production databases
Private network resources
Host-level resources
Container metadata
Shared storage

Even if the application itself is secure, an attacker may use the code execution feature as a path toward other resources.

For example, code such as this should never have access to your application's environment variables:

import os

for key, value in os.environ.items():
    print(key, value)

If the execution environment contains database credentials or API keys, those values can potentially become exposed.

Use a Separate Execution Boundary

The first design decision should be to separate the execution environment from the application that receives the request.

A typical architecture can look like this:

                    +----------------+
                    |     Client     |
                    +-------+--------+
                            |
                            v
                    +----------------+
                    |       API      |
                    +-------+--------+
                            |
                     Queue / Job
                            |
                            v
                  +-------------------+
                  | Execution Service |
                  +---------+---------+
                            |
                            v
                  +-------------------+
                  | Isolated Sandbox  |
                  |                   |
                  | Untrusted Code    |
                  +---------+---------+
                            |
                       Limited access
                            |
                +-----------+-----------+
                | Temporary Storage    |
                | Specific API Access  |
                +----------------------+

The important part is the boundary between the application and the code being executed.

The sandbox should not be treated as an extension of the main application container.

Why a Normal Container Is Not Always Enough

Containers provide process and filesystem isolation, but they should not automatically be treated as a complete security boundary for hostile workloads.

A container running arbitrary code may still have access to resources that the application does not intend to expose.

For example, a poorly configured workload might receive:

Environment variables
IAM credentials
Network access
Shared volumes
Service endpoints
Container metadata

The security model should therefore assume that the code running inside the sandbox may attempt to escape its intended boundaries.

A better approach is defense in depth.

This means combining:

Workload isolation
Least-privilege IAM
Network restrictions
Short execution timeouts
CPU and memory limits
Ephemeral storage
Minimal runtime images
Logging and monitoring
Input and output controls

No single control should be responsible for the entire security model.

Isolate the Workload With AWS Nitro-Based Sandboxing

For workloads that require a stronger isolation boundary, AWS Nitro-based sandboxing can be used to isolate untrusted workloads from the surrounding application environment.

The basic idea is that the untrusted workload runs inside a strongly isolated execution environment instead of sharing the same application runtime.

Conceptually:

Main Application
       |
       | Job
       v
Execution Manager
       |
       v
+----------------------+
| Nitro-based Sandbox  |
|                      |
| Untrusted Workload   |
| Temporary Files      |
| Limited Resources    |
+----------------------+

This is particularly useful for workloads where the code itself cannot be trusted.

Give the Sandbox a Separate IAM Role

IAM permissions should be designed around the workload rather than the application.

For example, suppose a code execution job only needs to read a specific object from S3.

Do not give the execution environment:

AmazonS3FullAccess

Instead, create a dedicated IAM role with only the required permission.

A simplified policy might look like:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject"
      ],
      "Resource": "arn:aws:s3:::my-execution-inputs/jobs/*"
    }
  ]
}

The principle is simple:

Sandbox permissions != Application permissions

The application may need access to databases, queues, configuration services, and other AWS resources.

The sandbox usually does not.

Keeping those permissions separate limits the damage if the untrusted workload is compromised.

Avoid Passing Application Secrets Into the Sandbox

One common mistake is to pass secrets through environment variables because it is convenient.

For example:

DATABASE_URL=...
API_KEY=...
JWT_SECRET=...
AWS_ACCESS_KEY_ID=...

A sandbox running arbitrary code should not receive these values unless they are explicitly required.

Instead, the application should communicate with the sandbox using controlled inputs and outputs.

For example:

Application
    |
    | Input JSON
    v
Sandbox
    |
    | Result JSON
    v
Application

The sandbox should receive only the data required to perform the job.

This makes the security boundary much easier to reason about.

Restrict Network Access

Network access is another important part of the isolation model.

An untrusted workload generally should not have unrestricted outbound access.

For example, if the code execution job only needs to process a local input file, it may not need Internet access at all.

A restrictive model could look like this:

Sandbox
   |
   +-- Allowed: Object storage
   |
   +-- Allowed: Specific internal API
   |
   +-- Blocked: Database
   |
   +-- Blocked: Metadata services
   |
   +-- Blocked: Internal administration endpoints
   |
   +-- Blocked: Unrestricted Internet

This is particularly important because a compromised process can use network connectivity to retrieve additional payloads or attempt to reach internal services.

Network isolation should therefore be designed as part of the execution architecture rather than added later.

Put Resource Limits on Every Job

Untrusted code can create another type of security problem: resource exhaustion.

A malicious or poorly written program could consume CPU indefinitely:

while True:
    pass

It could also allocate excessive memory:

data = []

while True:
    data.append("x" * 1024 * 1024)

The sandbox should therefore enforce limits such as:

Maximum execution time
Maximum CPU
Maximum memory
Maximum temporary storage
Maximum output size
Maximum number of concurrent jobs

For example:

Job timeout:       30 seconds
Memory limit:      512 MB
CPU limit:         1 vCPU
Storage limit:     100 MB
Output limit:      10 MB

These values should be based on the actual workload rather than copied directly into production.

Make the Execution Environment Ephemeral

A sandbox should ideally be disposable.

Instead of maintaining a long-lived environment:

Sandbox A
  Job 1
  Job 2
  Job 3
  Job 4

prefer a model where the environment is created for a workload and discarded afterward:

Job 1 -> Sandbox -> Destroy
Job 2 -> Sandbox -> Destroy
Job 3 -> Sandbox -> Destroy

This reduces the chance that data from one execution remains available to another execution.

Ephemeral environments are particularly useful when different users submit unrelated workloads.

Validate the Input Before Execution

Isolation does not remove the need for input validation.

The application should validate:

Maximum source-code size
Supported runtime
Input file size
Command parameters
Job timeout
Resource requirements
Output size

For example, an API should not allow a client to request:

{
  "timeout": 86400,
  "memory": "128GB",
  "cpu": 64
}

unless the system is intentionally designed to support those values.

The execution service should apply its own limits rather than trusting the client.

Capture Logs Without Exposing Sensitive Data

Execution logs are useful for debugging and security monitoring.

However, logging everything produced by untrusted code can create another security problem.

For example, user code might print:

print(os.environ)

If those logs are stored permanently, sensitive information could be exposed through the logging system.

A better logging design separates:

Execution metadata
    |
    +-- Job ID
    +-- Start time
    +-- Duration
    +-- Exit status
    +-- Resource usage

Application output
    |
    +-- Controlled
    +-- Size limited
    +-- Sanitized where appropriate

The execution platform should also make it possible to identify which user or application request created each job.

Common Mistakes

Reusing the Application Container

Running untrusted code directly inside the API container creates unnecessary risk.

The application container should remain responsible for application logic, not arbitrary user code.

Giving the Sandbox Broad IAM Permissions

A role such as AdministratorAccess defeats the purpose of isolation.

The execution role should contain only the permissions required by the workload.

Leaving Network Access Open

A sandbox with unrestricted outbound and internal connectivity provides an attacker with more opportunities to move beyond the execution environment.

Keeping Sandboxes Alive

Long-lived execution environments can retain files, processes, and state that should not survive between jobs.

Trusting Client-Provided Resource Limits

Resource limits should be enforced by the execution service.

A client should not be able to request unlimited CPU, memory, storage, or execution time.

A Practical Security Checklist

Before moving an untrusted execution system into production, verify the following:

Area	Recommended approach
Isolation	Use a dedicated sandbox boundary
IAM	Separate execution role
Secrets	Do not expose application secrets
Network	Allow only required destinations
Runtime	Use a minimal environment
CPU	Enforce a fixed or bounded limit
Memory	Enforce a bounded limit
Storage	Use temporary storage
Timeout	Kill jobs that exceed the limit
Output	Limit and control output size
Logging	Capture execution metadata safely
Lifecycle	Prefer ephemeral execution
Monitoring	Track failures and unusual resource usage

When This Architecture Makes Sense

Strong workload isolation is useful when an application executes code that it does not fully control.

Examples include:

Online code runners
AI-generated code execution
Plugin systems
Customer-defined scripts
Data transformation jobs
Build systems
Automated testing platforms
Document or file processing
Multi-tenant developer platforms

For completely trusted application code, this level of isolation may add unnecessary operational complexity.

The security boundary should match the trust boundary.

Final Thoughts

Running untrusted code safely in AWS requires more than putting the code inside a container.

The important design decision is to treat the workload as potentially hostile and limit what it can access.

For workloads that require stronger isolation, Nitro-based sandboxing can provide an additional boundary between untrusted code and the surrounding infrastructure.

The result is a system where even if an individual execution job behaves maliciously, the potential impact is limited to the resources that the sandbox was intentionally allowed to use.

AWSuntrusted codesandboxingAWS NitroIAM least privilegecontainer securityworkload isolation

Comments

Join the conversation! Your thoughts help the community grow.