Kafka consumer rebalancing is a normal part of running a consumer group. It happens when consumers join or leave the group, partitions are reassigned, or the group needs to recover from a failure.
The problem starts when rebalances happen too frequently.
A consumer can be doing useful work one moment and then lose its partition assignment. Messages may be processed again, consumption can pause, and lag can grow while the group settles on a new assignment.
For applications processing large volumes of events, frequent rebalancing can become a performance and reliability problem.
The goal is not to eliminate rebalancing. Kafka needs it to maintain consumer group membership and partition ownership. The goal is to make consumer behavior predictable during rebalances and prevent normal application work from causing unnecessary group instability.
This article explains why Kafka consumers rebalance, what causes avoidable rebalances, and how to design a C# consumer that handles them safely.
What Is a Kafka Consumer Rebalance?
Kafka divides a topic into partitions.
For example:
orders
├── Partition 0
├── Partition 1
├── Partition 2
└── Partition 3A consumer group processes those partitions.
With two consumers:
Consumer A -> Partition 0, Partition 1
Consumer B -> Partition 2, Partition 3If Consumer B leaves the group, Kafka needs to redistribute its partitions.
Consumer A -> Partition 0, Partition 1, Partition 2, Partition 3That redistribution is a rebalance.
A rebalance can happen when:
A new consumer joins the group.
A consumer shuts down.
A consumer crashes.
A consumer misses group heartbeats.
Partition assignments change.
A consumer takes too long between polls.
Deployment replaces consumer instances.
Group membership changes.
Rebalancing itself is expected behavior.
The problem is frequent or poorly handled rebalancing.
Why Rebalances Can Affect Application Performance
During a rebalance, partition ownership changes.
A simplified sequence looks like this:
Consumer A
|
| Processing
v
Kafka detects membership change
|
v
Rebalance starts
|
v
Partition ownership changes
|
v
Consumers resume processingDepending on the consumer configuration and assignment strategy, consumption can pause during this process.
If the application is already under heavy load, even a short pause can increase consumer lag.
Repeated rebalances can create a cycle:
Slow processing
|
v
Consumer misses expected timing
|
v
Rebalance
|
v
Consumption pauses
|
v
Lag increases
|
v
More work accumulates
|
v
Consumer becomes slower
|
v
Another rebalanceThis is why consumer stability matters as much as raw processing speed.
The Most Common Cause Is Consumer Timing
Kafka needs to know that a consumer is still participating in the group.
Two timing concepts are especially important:
Heartbeat behavior
Poll interval
The exact configuration depends on the Kafka client and workload, but the underlying principle is simple.
The consumer must continue communicating with Kafka while processing records.
A consumer that spends too long doing synchronous work without polling can be considered unhealthy.
For example:
Poll
|
v
Process 10,000 records
|
| 10 minutes
v
Poll againIf the processing interval exceeds the configured limits, Kafka may remove the consumer from the group.
The result is a rebalance.
Separate Polling From Slow Business Work
One of the most useful design improvements is separating Kafka consumption from slow business processing.
Instead of making the Kafka consumer directly perform a long-running operation:
Kafka Consumer
|
v
Database
|
v
External API
|
v
Large computationuse a controlled processing pipeline:
Kafka Consumer
|
v
Bounded Work Queue
|
+---- Worker 1
+---- Worker 2
+---- Worker 3The consumer can continue participating in the group while workers process the application workload.
However, this design introduces another responsibility.
You must not allow the internal queue to grow without limits.
A bounded queue provides backpressure.
Using a Bounded Channel in C#
The .NET Channel<T> class works well for this type of pipeline.
For example:
var channel = Channel.CreateBounded<OrderEvent>(
new BoundedChannelOptions(500)
{
FullMode = BoundedChannelFullMode.Wait,
SingleWriter = true,
SingleReader = false
});The Kafka consumer writes records into the channel:
await channel.Writer.WriteAsync(
orderEvent,
cancellationToken);Worker tasks can then process the events:
await foreach (var orderEvent in
channel.Reader.ReadAllAsync(cancellationToken))
{
await ProcessOrderAsync(
orderEvent,
cancellationToken);
}The bounded capacity prevents Kafka consumption from creating unlimited in-memory work.
Do Not Process Unlimited Records in Parallel
A common response to consumer lag is increasing concurrency.
For example:
Consumer
|
+--> Task 1
+--> Task 2
+--> Task 3
+--> ...
+--> Task 1000This can make the problem worse.
If every task performs a database query, the application may overwhelm the database.
If every task calls an external API, the API may throttle the application.
If memory usage increases, the process can become unstable.
A better design uses controlled concurrency:
Kafka Consumer
|
v
Bounded Queue
|
+--> Worker 1
+--> Worker 2
+--> Worker 3
+--> Worker 4The correct number of workers should be based on the workload and downstream capacity.
Understand max.poll.interval.ms
Kafka clients expose a configuration that controls the maximum interval between polls.
If your consumer performs too much work between polls, it can exceed this interval.
For a .NET Kafka client, the configuration may look like:
var config = new ConsumerConfig
{
BootstrapServers = bootstrapServers,
GroupId = "orders-service",
AutoOffsetReset = AutoOffsetReset.Earliest,
EnableAutoCommit = false,
MaxPollIntervalMs = 300000
};The important point is not to simply increase the value whenever rebalances occur.
If processing genuinely takes longer, you should first understand why.
Increasing the interval can hide an underlying performance problem.
For example:
Database query = 2 seconds
Message processing = 3 seconds
External API = 8 seconds
Total = 13 secondsIf the consumer processes many records in one poll before polling again, the total time can become much larger than expected.
Reduce the Amount of Work Per Poll
Another useful configuration is the maximum number of records returned in one poll.
For example:
var config = new ConsumerConfig
{
BootstrapServers = bootstrapServers,
GroupId = "orders-service",
MaxPollIntervalMs = 300000,
MaxPollRecords = 100
};If processing each record takes time, reducing the number of records processed per polling cycle can help keep the consumer responsive.
The right value depends on:
Processing time per record
Database capacity
External API latency
Consumer CPU usage
Message size
Desired throughput
There is no universal value that works for every application.
Offset Commit Strategy Matters
Kafka consumers need to commit offsets to track progress.
There are two broad concerns:
Message received
|
v
Message processed
|
v
Offset committedThe order matters.
If an offset is committed before the business operation completes, a crash can result in lost processing.
If the business operation completes but the offset is not committed before a crash, Kafka can deliver the record again.
That second scenario is why consumers should be idempotent.
A safe mental model is:
Process successfully
|
v
Commit offsetBut even this does not create exactly-once business behavior by itself.
The database operation and Kafka offset commit are separate systems.
Why Duplicate Processing Can Still Happen
Consider this sequence:
1. Kafka delivers message 100
2. Consumer processes message 100
3. Database transaction succeeds
4. Consumer crashes
5. Offset 100 was not committed
6. Kafka delivers message 100 againThe message is processed twice.
This is normal for at-least-once processing.
The application should make important operations idempotent.
For example:
eventId = 7b8e...
First processing
-> Database change
-> Record eventId
Duplicate processing
-> eventId already exists
-> Skip business operationThis is particularly important when rebalances are frequent.
Cooperative Rebalancing Can Reduce Disruption
Kafka supports different partition assignment approaches.
Some strategies can move many partitions away from consumers during a rebalance.
Cooperative rebalancing allows partition movement to happen more gradually.
With the Confluent Kafka .NET client, the configuration can be selected through the partition assignment strategy.
For example:
var config = new ConsumerConfig
{
BootstrapServers = bootstrapServers,
GroupId = "orders-service",
PartitionAssignmentStrategy =
PartitionAssignmentStrategy.CooperativeSticky
};The exact behavior depends on the Kafka client version and broker/client compatibility, so configuration changes should be tested in a staging environment.
Join the conversation! Your thoughts help the community grow.