Kubernetes resource limits are often treated as a simple safety setting: give a container a memory limit, set a CPU limit, and Kubernetes will prevent the application from using more resources than expected. In practice, memory limits can behave very differently from what developers expect.
A pod can appear to have a reasonable memory limit and still get terminated with an OOMKilled status. This usually happens because the configured limit does not match the application's actual memory behavior. A container may have short memory spikes, multiple processes, native allocations, large temporary objects, or workload patterns that are not visible when looking only at normal memory usage.
Understanding the difference between requests, limits, actual memory usage, and Kubernetes' eviction behavior makes these problems much easier to diagnose.
Kubernetes Resource Requests and Limits
Kubernetes allows CPU and memory resources to be configured for containers through requests and limits.
A request tells Kubernetes how much resource the container normally needs. The scheduler uses this value when deciding where a pod can run.
A limit defines the maximum amount of a resource that a container is allowed to consume. For memory, reaching this limit can result in the container being terminated by the Linux out-of-memory mechanism.
A basic configuration looks like this:
resources:
requests:
cpu: "250m"
memory: "256Mi"
limits:
cpu: "500m"
memory: "512Mi"
Here, the container requests 256 MiB of memory and has a maximum memory limit of 512 MiB.
The important part is that the limit is not a prediction of how much memory the application will use. It is an enforced boundary.
If the application normally uses 300 MiB but occasionally needs 600 MiB, a 512 MiB limit can still cause an OOMKill even though the application's average memory usage looks completely healthy.
Why a Safe-Looking Limit Can Still Cause an OOMKill
The most common mistake is choosing a memory limit based on normal or average usage.
For example, imagine an API that normally uses around 350 MiB:
Normal usage: 300–350 MiB
Busy periods: 400–450 MiB
Memory spike: 600 MiB
Configured limit: 512 MiB
Monitoring may show a healthy application most of the time. However, when a request causes a temporary memory spike above 512 MiB, the container can be killed.
This is particularly common with operations such as:
Large JSON serialization or deserialization
File processing
Image or document processing
Large database result sets
Export generation
In-memory caching
Large object allocations
Parallel request processing
Temporary buffers
Native libraries
Runtime garbage collection
The application does not have to permanently consume more memory than the limit. A short-lived peak can be enough.
Memory Request and Memory Limit Are Not the Same Thing
A common configuration mistake is assuming that requests and limits represent the same type of guarantee.
Consider:
resources:
requests:
memory: "256Mi"
limits:
memory: "512Mi"
The request tells the scheduler that this pod needs 256 MiB of memory for placement decisions.
The limit tells the container runtime and operating system that the container cannot continue consuming memory beyond the configured boundary.
This means a pod can legitimately use more than its request.
For example:
Request: 256 MiB
Actual usage: 420 MiB
Limit: 512 MiB
There is nothing inherently wrong with this.
The problem starts when the application's real workload occasionally exceeds the limit:
Request: 256 MiB
Actual usage: 620 MiB
Limit: 512 MiB
At that point, the container is vulnerable to an OOMKill.
OOMKilled vs Kubernetes Eviction
These two situations are often confused because both can result in a pod disappearing or restarting.
An OOMKill usually means a process inside the container was terminated because the system ran out of memory according to its memory constraints.
You can often see this through:
kubectl describe pod <pod-name>
Look for a container state similar to:
Last State:
Terminated:
Reason: OOMKilled
You can also inspect the pod directly:
kubectl get pod <pod-name> -o json
An eviction is different. Kubernetes may evict pods when a node is under resource pressure, particularly when available memory becomes critically low.
For example, a node might have:
Node memory: 16 GiB
Available memory: very low
Several pods: consuming memory
The kubelet can begin evicting pods to protect the node.
So the first troubleshooting question should be:
"Was the container killed because it exceeded its memory limit, or was the pod evicted because the node was under memory pressure?"
The answer changes the investigation.
Check the Actual Container State First
When a pod restarts unexpectedly, do not immediately increase the memory limit.
Start with the pod status:
kubectl get pod <pod-name>
Then inspect the full details:
kubectl describe pod <pod-name>
Look for:
State
Last State
Reason
Exit Code
Restart Count
Events
If the previous container terminated with OOMKilled, that is strong evidence that memory usage exceeded the available container memory constraint.
You can also check the previous container logs:
kubectl logs <pod-name> --previous
This is useful because the current container may have already restarted. The logs from the previous process can reveal what the application was doing immediately before termination.
Container Memory Is More Than Managed Heap Memory
Another common mistake is comparing Kubernetes memory usage only with the application's managed heap.
For example, a .NET application may show a managed heap size that looks comfortably below the Kubernetes memory limit.
That does not necessarily mean the process is safe.
A process can consume memory through several areas, including:
The same general principle applies to applications built with other runtimes.
If Kubernetes reports that a container is approaching its memory limit, looking only at one runtime metric can give an incomplete picture.
This is why application-level memory metrics and container-level memory metrics should be monitored together.
Large Requests Can Create Temporary Memory Spikes
One of the easiest ways to produce an unexpected OOMKill is processing large amounts of data in memory.
Consider an API that accepts a large JSON payload.
A simplified application flow might look like:
HTTP request
↓
Request body
↓
JSON deserialization
↓
Application objects
↓
Validation
↓
Processing
↓
Response serialization
During processing, the application may temporarily hold multiple representations of the same data.
For example, a large payload may exist as request buffers while deserialized objects are also created in memory.
The final memory usage can therefore be much higher than the size of the original HTTP payload.
This becomes more dangerous when several large requests are processed concurrently.
If one request requires 150 MiB of temporary memory and four requests execute at the same time, the application can experience a significant memory spike even though each individual request looks manageable.
CPU Limits Can Indirectly Affect Memory Behavior
CPU and memory are different resources, but CPU configuration can still affect application behavior.
Suppose an application processes requests slowly because it has very limited CPU. Requests may remain active longer, increasing the number of objects that remain alive at the same time.
A queue of pending work can also grow when processing is slower than incoming traffic.
For example:
Incoming requests: 100/sec
Processing capacity: 70/sec
If the application retains request-related data while waiting for processing, memory usage can gradually increase.
Eventually, the problem may appear to be a memory problem even though the original bottleneck was insufficient processing capacity.
This is why memory troubleshooting should also consider throughput, concurrency, CPU usage, and queue depth.
Garbage Collection Does Not Guarantee Protection From OOMKills
Developers sometimes assume that garbage collection will automatically prevent memory limits from being exceeded.
Garbage collection can reclaim objects that are no longer reachable, but it cannot solve every memory problem.
If the application continues creating objects faster than they can be reclaimed, memory usage can keep increasing.
A workload can also temporarily require more memory before garbage collection has an opportunity to reclaim unused objects.
For example:
Request starts
↓
Large objects allocated
↓
More requests arrive
↓
Memory rises
↓
GC runs
↓
Some memory is reclaimed
↓
New allocations continue
If the process reaches its container memory limit during this cycle, the operating system can terminate it before the application gets an opportunity to recover.
The correct solution is therefore not simply "increase the memory limit." You need to understand why the allocation pattern exists.
How to Choose a Better Memory Limit
A practical approach is to measure actual memory usage under realistic workloads.
Do not use only the average.
Look at:
Normal memory usage
Peak memory usage
Usage during deployments
Usage during traffic spikes
Usage during large requests
Usage after garbage collection
Memory usage under concurrency
Memory usage during background processing
For example:
Normal: 300 MiB
Typical peak: 430 MiB
Busy peak: 500 MiB
Rare peak: 620 MiB
Limit: 512 MiB
This configuration is clearly vulnerable to the rare peak.
Instead of blindly doubling the limit, first investigate what causes the 620 MiB spike.
If the spike is expected and unavoidable, the limit may need to accommodate it.
If the spike is caused by an inefficient operation, fixing the application may be better than allocating more memory.
Common Configuration Mistakes
Setting Limits From Developer Machines
A developer machine may have several gigabytes of available memory, while the production container has a much smaller limit.
An operation that appears harmless locally can therefore fail in production.
Test important workloads under resource constraints that resemble production.
Using the Same Limit for Every Pod
Different workloads have different memory profiles.
A lightweight API, background worker, message consumer, and report-generation service may require very different limits.
Resource settings should be based on workload behavior rather than copied across deployments.
Increasing Limits Without Finding the Cause
Increasing a limit can hide a memory leak or inefficient processing pattern.
If a container moves from:
512 MiB → 1 GiB → 2 GiB
and continues growing over time, the real problem is probably not the original limit.
Ignoring Restart Counts
A pod that appears healthy may actually be restarting repeatedly.
Check:
kubectl get pods
and look at:
RESTARTS
A high restart count combined with OOMKilled states is a strong signal that memory behavior needs investigation.
A Practical Troubleshooting Checklist
When a container is repeatedly OOMKilled, follow this order:
Check the pod status.
kubectl get pod <pod-name>
Inspect the previous termination reason.
kubectl describe pod <pod-name>
Check logs from the previous container.
kubectl logs <pod-name> --previous
Check current memory usage.
kubectl top pod <pod-name>
Compare actual usage with the configured limit.
kubectl get pod <pod-name> -o yaml
Check whether memory usage increases over time.
Look for large requests, batch operations, caches, or background tasks.
Check concurrency and traffic during the failure.
Check node memory pressure if the pod was evicted instead of OOMKilled.
Fix the workload or adjust the resource configuration based on measured behavior.
This sequence prevents a common mistake: changing Kubernetes configuration before understanding the application's memory pattern.
Best Practices for Production
Use measured resource requests and limits rather than arbitrary values.
Monitor both container memory and application-level memory metrics.
Test memory behavior under realistic concurrency instead of testing only one request at a time.
Pay attention to temporary memory spikes, not only steady-state usage.
Set alerts before the container reaches its limit so the team has time to investigate.
For memory-heavy operations, consider streaming instead of loading entire datasets into memory.
Review memory settings after significant application changes, such as introducing a new caching layer, changing serialization, increasing concurrency, or adding background processing.
Most importantly, treat an OOMKill as a symptom to investigate rather than simply a number to fix.
Summary
A memory limit that looks safe can still cause OOMKills when an application has short-lived memory spikes, high concurrency, large allocations, or unexpected workload growth.
Start by checking whether the container was actually OOMKilled or the pod was evicted because of node pressure. Then compare container memory usage with the configured limit and investigate what caused the spike.
The right memory limit comes from measured application behavior. Increasing the limit can help when the workload genuinely needs more memory, but it should not replace fixing inefficient memory usage or leaks.
Join the conversation! Your thoughts help the community grow.