Kubernetes 1.37 etcd RangeStream and Its Impact on API Server Memory

Large Kubernetes clusters can put significant memory pressure on the control plane when the API server needs to read a large collection of objects from etcd.

This becomes especially visible when the API server initializes or rebuilds its watch cache. A cluster with many Pods, large Custom Resources, or other high-volume resources can require the API server and etcd to process a large amount of data at the same time.

Kubernetes 1.37 introduces Beta support for etcd RangeStream. With etcd 3.7 or later, the API server can stream large result sets in chunks instead of receiving one large buffered response.

The goal is simple: reduce peak memory usage and make large reads more predictable.

Why Large etcd Reads Consume So Much Memory

Kubernetes stores its persistent API state in etcd.

When the API server needs a collection of objects, it ultimately needs to read that data from etcd when the data is not already available from its watch cache.

The traditional etcd Range operation builds the complete response before returning it.

Consider a large collection:

etcd
 |
 |-- Object 1
 |-- Object 2
 |-- Object 3
 |-- ...
 `-- Object 100000
        |
        v
Build complete response
        |
        v
Send response
        |
        v
API server decodes response

The complete response can require substantial memory.

During processing, multiple representations of the same data can exist at different stages:

etcd
 |
 +-- Stored values
 |
 +-- Range response
 |
 +-- Serialized response
 |
 `-- gRPC send buffers
          |
          v
     API server
          |
          +-- Received response
          +-- Decoded objects
          `-- Watch cache data

This is why a large read can create a temporary memory spike even when the final Kubernetes objects would fit comfortably in memory.

Why Pagination Does Not Fully Solve the Problem

The API server already uses pagination in several read paths.

Instead of requesting every object at once, it can request a limited number of keys:

Page 1 -> 1,000 objects
Page 2 -> 1,000 objects
Page 3 -> 1,000 objects
...

That helps, but the page size is based primarily on the number of keys.

It does not know how large each object is.

Consider two pages:

Page A
1,000 small objects
= relatively small response

Page B
1,000 large objects
= very large response

Both pages contain the same number of keys, but their memory requirements can be dramatically different.

Large objects such as Pods with large specifications, CRDs, or other resource types can therefore make a supposedly bounded page much larger than expected.

There is another cost associated with paginated range operations in etcd. Each page can require work against the underlying key index to determine the total result count.

RangeStream changes the way this large collection is delivered.

What Is etcd RangeStream?

etcd 3.7 introduces a server-streaming RangeStream RPC.

It accepts the same basic RangeRequest concept as the traditional Range operation but returns the results in chunks.

The difference looks like this:

Traditional Range

etcd
 |
 |-- Build entire response
 |
 `---------------> API server


RangeStream

etcd
 |
 |-- Chunk 1 ------> API server
 |-- Chunk 2 ------> API server
 |-- Chunk 3 ------> API server
 |-- Chunk 4 ------> API server
 `-- ...

The server can adapt the chunk size based on the values being returned.

Large objects can therefore result in smaller chunks, while collections of small objects can be processed more efficiently.

The API server processes each chunk as it arrives and can release that chunk before receiving the next one.

How Kubernetes 1.37 Uses RangeStream

Kubernetes 1.37 adds the EtcdRangeStream feature gate.

It is Beta and enabled by default.

When the feature is active and the connected etcd server supports RangeStream, the API server can use streaming reads for large collections.

The main consumer is watch cache initialization.

The process becomes:

etcd
 |
 | RangeStream
 |
 +-- Chunk 1
 |     |
 |     v
 |   Decode
 |
 +-- Chunk 2
 |     |
 |     v
 |   Decode
 |
 +-- Chunk 3
 |     |
 |     v
 |   Decode
 |
 `-- ...

The API server does not need to assemble the entire collection in memory before beginning to process it.

This is the central memory improvement.

Watch Cache Initialization

The API server maintains an in-memory watch cache for many Kubernetes resources.

The cache allows normal list and watch operations to be served efficiently without querying etcd for every request.

When the API server starts or a watch cache needs to be rebuilt, it has to populate that cache from etcd.

For a small cluster, this is usually uninteresting.

For a large cluster, it can be expensive.

For example:

Cluster
|
+-- 5,000 nodes
|
+-- Hundreds of thousands of Pods
|
+-- Large CRDs
|
`-- Other high-volume resources

The API server may need to read and decode a very large amount of data before the cache is ready.

RangeStream allows the API server to start processing the result while etcd is still streaming the remaining data.

Memory Behavior Before and After

The difference can be simplified like this.

Traditional Range

Memory
 ^
 |
 |        ###########
 |        ###########
 |        ###########
 |        ###########
 |________###########
 |
 +----------------------> Time
          large peak

The complete response has to exist before processing can proceed.

RangeStream

Memory
 ^
 |
 |     ####
 |     ####    ####
 |     ####    ####    ####
 |_____##_##___####____####____
 |
 +-----------------------------> Time
       chunks processed incrementally

The goal is not to make memory consumption zero.

The API server still needs memory for decoded objects and its watch cache.

The improvement is that temporary buffers for the large etcd response do not have to grow into one large payload.

RangeStream Also Helps etcd Memory

The memory improvement is not limited to kube-apiserver.

With the traditional unary Range RPC, etcd builds the result before sending it.

That means etcd can temporarily hold:

Key-value results
Serialized response data
gRPC buffers
Other processing structures

With RangeStream, etcd can produce smaller chunks and release data as the stream progresses.

This is especially useful because etcd is also responsible for maintaining the cluster's persistent state.

A memory spike in etcd can affect more than the request that caused it.

What Happens With Older etcd Versions?

Kubernetes 1.37 requires etcd 3.7 or later to actually use RangeStream.

The API server checks whether the etcd server supports the new RPC.

If the server responds with Unimplemented, the API server falls back to the existing paginated Range behavior.

That means an API server can remain operational when paired with an older etcd version, although it will not receive the RangeStream memory benefits.

The behavior can be represented as:

kube-apiserver
      |
      v
Does etcd support RangeStream?
      |
      +-- Yes --> Stream results
      |
      `-- No ---> Use paginated Range

This fallback is useful during controlled upgrades.

How to Verify RangeStream Is Being Used

Kubernetes exposes etcd request metrics that identify streamed reads.

The relevant operation label is:

operation="listStream"

For example:

etcd_request_duration_seconds_count{operation="listStream"}

A non-zero count indicates that the API server is using RangeStream.

If the count remains zero, investigate the etcd version and feature-gate configuration.

The most common reason is that the API server is connected to an etcd version that does not implement the required RPC.

Feature Gate Configuration

EtcdRangeStream is enabled by default in Kubernetes 1.37.

If you need to disable it temporarily:

--feature-gates=EtcdRangeStream=false

Disabling the feature returns the API server to its previous paginated Range behavior.

This can be useful for troubleshooting or when an etcd-compatible backend or proxy does not correctly handle the streaming RPC.

If you operate an etcd-compatible backend rather than standard etcd, test the behavior before enabling the feature across the cluster.

Direct List Requests Also Benefit

Watch cache initialization is the main use case, but it is not the only one.

The API server can also use RangeStream when it needs to read a collection directly from etcd.

This matters when a list request cannot be served through the normal watch-cache path.

The general flow is:

Client
  |
  v
kube-apiserver
  |
  +-- Watch cache available?
  |       |
  |       `-- Yes --> Serve from cache
  |
  `-- No
        |
        v
    etcd RangeStream
        |
        v
    Process chunks

The benefit is therefore broader than just startup behavior.

Large Objects Matter More Than Object Count

One of the most useful operational lessons from RangeStream is that object count alone is a poor way to estimate memory pressure.

Suppose a resource contains:

100,000 small objects

That may be manageable.

Another resource could contain:

20,000 large objects

and create a much larger temporary memory requirement.

A page containing 1,000 large objects can be more expensive than several pages containing thousands of small objects.

When investigating API server memory spikes, look at both:

Number of objects
Size of objects

This is particularly important for CRDs.

CRDs and Large API Objects

Custom Resources can vary widely in size.

A CRD used as an application configuration object might contain:

spec:
  configuration:
    ...
    large nested structure

If thousands of such objects exist, a list operation can become expensive.

RangeStream reduces the buffering overhead of reading those objects from etcd.

It does not make large objects free.

The objects still need to be decoded and stored where Kubernetes requires them.

That means reducing object size remains a good operational practice.

RangeStream Does Not Increase Watch Cache Capacity

It is important to understand what RangeStream does not change.

The API server still needs memory for its watch cache.

If a resource has a very large number of objects, the cache itself can consume substantial memory.

RangeStream mainly reduces temporary memory associated with reading the collection from etcd.

Think of the change as:

Before

Persistent storage
     |
     v
Large temporary read
     |
     v
Decode
     |
     v
Watch cache


After

Persistent storage
     |
     v
Small stream chunks
     |
     v
Decode incrementally
     |
     v
Watch cache

The final watch cache remains.

Common Mistakes

Assuming RangeStream Fixes All API Server OOM Problems

It does not.

API server memory can also be affected by:

Watch cache size
Concurrent requests
Large API objects
Admission processing
Serialization
Client behavior
Controllers creating excessive objects

RangeStream addresses a specific part of the memory problem.

Assuming Pagination and Streaming Are the Same

They are not.

Pagination limits the number of keys returned by each request.

RangeStream streams the response and adapts chunk sizes based on the data being returned.

Checking Only CPU and Memory of kube-apiserver

The etcd side also matters.

When investigating large-list performance, monitor both components.

Forgetting Custom Resources

Operators often focus on built-in resources such as Pods and Nodes.

Large CRDs can also generate substantial API server and etcd load.

Include them when investigating large collection reads.

Ignoring the etcd Version

Kubernetes 1.37 can run with older etcd versions through fallback behavior, but RangeStream itself requires etcd 3.7 or later.

If listStream remains at zero, check the etcd version first.

Troubleshooting Checklist

RangeStream Is Not Being Used

Check the etcd version:

etcd --version

Then inspect the API server metrics:

etcd_request_duration_seconds_count{operation="listStream"}

If the metric remains zero, confirm that the feature gate has not been disabled.

API Server Memory Is Still High

Check whether the problem is temporary buffering or persistent watch-cache memory.

RangeStream does not reduce the memory required to retain the cache itself.

Also inspect object sizes and resource counts.

etcd Memory Is Still High

Look at:

Large resources
Concurrent list operations
Watch cache initialization
Client request patterns
etcd database size
Compaction and maintenance behavior

RangeStream reduces memory needed for a large range response, but it does not remove the underlying data or other etcd workloads.

A Proxy or Compatible Backend Causes Errors

If Kubernetes is using an etcd-compatible backend or proxy, verify that it handles the streaming RPC correctly.

If necessary, disable the feature temporarily:

--feature-gates=EtcdRangeStream=false

Operational Recommendations

Upgrade etcd Before Expecting the Benefit

Kubernetes 1.37 can fall back to older etcd versions, but the RangeStream path requires etcd 3.7 or later.

Plan the etcd upgrade as part of the control-plane rollout.

Monitor Both Sides

At minimum, monitor:

kube-apiserver memory
etcd memory
etcd request latency
etcd request operation type
watch cache initialization
API server restarts

Track Large Resources

Identify resource types with:

High object counts
Large object sizes
Frequent list operations
Large watch caches

This gives you a better picture of where control-plane memory is actually going.

Review Dashboards

Existing monitoring dashboards may classify etcd requests by:

operation="list"

RangeStream uses:

operation="listStream"

Update dashboards and alerts so streamed reads are not accidentally excluded from existing list-request monitoring.

Range vs RangeStream

Area	etcd Range	RangeStream
Response model	Buffered response	Streaming response
Large result sets	Higher peak buffering	Lower peak buffering
Chunking	Client/API-server pagination	Server-side streaming
Object-size awareness	Limited by key-based pages	Adaptive chunk sizing
Kubernetes 1.37	Existing path	Beta
etcd requirement	Existing supported versions	etcd 3.7+
API server fallback	Not applicable	Falls back when unsupported
Best use case	Normal reads	Large collections

Advantages and Disadvantages

Advantages

Reduces peak memory used for large etcd reads.
Reduces temporary buffering on the API server and etcd.
Makes memory usage more predictable for large collections.
Helps watch cache initialization.
Handles large object sizes better than simple key-count pagination.
Falls back to the existing path when the backend does not support the RPC.

Disadvantages

Requires etcd 3.7 or later to provide the full benefit.
Does not reduce the persistent memory required by the watch cache.
Does not solve all API server or etcd memory problems.
Adds another request operation that monitoring systems need to understand.
Compatible etcd proxies and alternative backends need testing.

A Practical Upgrade and Validation Plan

For an existing Kubernetes 1.37 cluster, validate RangeStream in stages.

Check the etcd version.

etcd --version

Confirm the feature gate.

EtcdRangeStream

It should be enabled by default in Kubernetes 1.37 unless your configuration overrides it.

Monitor API server memory.

Establish a baseline before the change.

Check streamed request metrics.

Look for:

operation="listStream"

Observe etcd memory.

The improvement should be evaluated on both sides of the API boundary.

Test during cache initialization.

Restarting an API server in a controlled environment can help expose the difference because the watch cache needs to rebuild.

Review large resources.

Check Pods and high-volume CRDs for unusually large object sizes.

Update monitoring.

Make sure dashboards and alerts include both list and listStream operations.

Final Checklist

Before relying on RangeStream in a Kubernetes 1.37 control plane, verify:

Check	What to confirm
Kubernetes	Running Kubernetes 1.37 or later
etcd	Running etcd 3.7 or later
Feature gate	`EtcdRangeStream` is enabled
Metrics	`listStream` requests are visible
API server	Memory usage is being monitored
etcd	Memory and latency are being monitored
CRDs	Large custom resources have been reviewed
Dashboards	Streamed requests are included
Proxy	Compatible etcd proxies have been tested
Fallback	Existing Range behavior works if streaming is unavailable

Conclusion

Kubernetes 1.37's etcd RangeStream support addresses a specific control-plane problem: large collection reads can require too much temporary memory when the entire result is buffered before processing.

With etcd 3.7, RangeStream lets kube-apiserver receive the result incrementally. The API server can decode each chunk, process it, and release the temporary data before receiving the next chunk.

The change is especially useful for large watch-cache initialization and other list operations that need to read substantial collections from etcd.

It does not eliminate the memory required by the watch cache, and it does not solve every control-plane memory problem. The practical approach is to combine RangeStream with monitoring of object counts, object sizes, API server memory, and etcd behavior.

For large Kubernetes clusters, that makes RangeStream a useful improvement in controlling the memory cost of large reads.