RabbitMQ Prefetch Count: Why Your Consumers Process Messages Unevenly

When a RabbitMQ queue contains thousands of messages, you might expect multiple consumers to share the workload evenly. However, one consumer may process hundreds of messages while another remains almost idle.

This can happen even when all consumers are connected, healthy, and listening to the same queue.

One setting that affects this behavior is the prefetch count. It controls how many unacknowledged messages RabbitMQ can deliver to a consumer at a time, subject to the configured quality-of-service settings and the consumer's state.

A poorly chosen prefetch count can cause uneven workloads, slow message processing, and unnecessary memory usage. A suitable value can improve throughput while allowing consumers to share work more effectively.

In this article, we will look at how RabbitMQ prefetch works, how to configure it in C#, and how to troubleshoot consumers that process messages unevenly.

What Is RabbitMQ Prefetch Count?

RabbitMQ delivers messages to consumers over channels. When a consumer receives a message, it can acknowledge the message after processing it.

Until the message is acknowledged, it remains unacknowledged. With manual acknowledgements, the prefetch count limits how many messages can be outstanding at once under the applicable QoS configuration.

For example, suppose a queue has 1,000 messages and two consumers.

Consumer A processes messages quickly.
Consumer B processes messages more slowly.
The prefetch count is set to 100.

RabbitMQ may deliver a large batch of messages to Consumer A before those messages have been acknowledged. If Consumer A is slow to finish that batch, messages already assigned to it cannot simply be redistributed to Consumer B.

This can lead to uneven processing even when both consumers are available.

The prefetch count is not the number of messages a consumer processes simultaneously. It is a limit on outstanding, unacknowledged deliveries. Actual processing concurrency depends on the consumer implementation.

How Prefetch Affects Message Distribution

Consider three consumers processing messages from the same queue.

Prefetch count	Typical behavior	Possible drawback
1	Each consumer generally receives another message after acknowledging its current delivery	More delivery and acknowledgement coordination can reduce throughput
10	Consumers can work on several outstanding messages	Some imbalance may remain if processing times vary
100	Consumers can have large batches of messages assigned to them	Fast consumers may hold work that slower consumers could process
Unlimited or no effective limit	Many deliveries may become outstanding	Memory pressure and large unacknowledged backlogs can develop

These are general tendencies, not guaranteed outcomes. The best setting depends on message size, processing duration, acknowledgement behavior, consumer concurrency, and RabbitMQ version.

A lower prefetch count can improve workload sharing when messages take different amounts of time to process. A higher count can improve throughput when message handling is fast and predictable.

The goal is to balance throughput, fairness, and resource usage.

Step 1: Understand Manual Acknowledgements

Prefetch is most useful when consumers use manual acknowledgements.

In manual acknowledgement mode, the consumer tells RabbitMQ that a delivery has been processed successfully.

With automatic acknowledgement, RabbitMQ considers a message acknowledged when it delivers it to the consumer. The broker therefore cannot use the consumer's later processing completion to limit outstanding work in the same way.

For reliable processing, a consumer often uses manual acknowledgements:

Receive a message.
Validate and process it.
Commit required database changes.
Acknowledge the message.

If processing fails, the consumer can reject or negatively acknowledge the delivery according to the application's retry policy.

Acknowledging before the business operation is safely completed can cause message loss if the consumer crashes afterward.

Step 2: Configure Prefetch in C#

The following example uses the RabbitMQ .NET client API style available in recent versions of the client. Check the API for your installed package version, because connection and consumer APIs have changed across releases.

The essential configuration is the channel's QoS setting:

await channel.BasicQosAsync(
    prefetchSize: 0,
    prefetchCount: 10,
    global: false);

Here:

A prefetch size of 0 means no byte-based limit is applied by this setting.
A prefetch count of 10 limits outstanding unacknowledged deliveries to the applicable consumer scope.
A global value of false applies the count to consumers rather than setting one shared limit for the entire channel in the classic QoS behavior.

For many consumer applications, a small positive prefetch count is a useful starting point.

A prefetch count of 10 does not mean that the consumer will run ten message handlers at the same time. If the application processes messages sequentially, it may receive several deliveries but handle them one after another.

Step 3: Build a Consumer with Manual Acknowledgements

The following example shows the core processing logic. It assumes that a channel and queue have already been created and that the consumer is registered with that channel.

using RabbitMQ.Client;
using RabbitMQ.Client.Events;
using System.Text;

await channel.BasicQosAsync(
    prefetchSize: 0,
    prefetchCount: 10,
    global: false);

var consumer = new AsyncEventingBasicConsumer(channel);

consumer.ReceivedAsync += async (_, eventArgs) =>
{
    try
    {
        var message = Encoding.UTF8.GetString(
            eventArgs.Body.ToArray());

        await ProcessMessageAsync(message);

        await channel.BasicAckAsync(
            eventArgs.DeliveryTag,
            multiple: false);
    }
    catch (Exception ex)
    {
        Console.Error.WriteLine(
            $"Message processing failed: {ex.Message}");

        await channel.BasicNackAsync(
            eventArgs.DeliveryTag,
            multiple: false,
            requeue: false);
    }
};

await channel.BasicConsumeAsync(
    queue: "orders",
    autoAck: false,
    consumer: consumer);

The example demonstrates manual acknowledgement and prefetch configuration, but a production consumer needs additional error handling.

In particular, requeueing every failed message immediately can create a tight retry loop. Rejecting a message without requeueing may also discard it unless the queue is configured with an appropriate dead-letter strategy.

Use a defined retry policy, and route messages that repeatedly fail to a dead-letter queue or another controlled failure workflow.

Also, do not assume that every RabbitMQ .NET client version permits concurrent operations on the same channel. Follow the installed client's concurrency requirements, especially if multiple message handlers can run concurrently.

Step 4: Choose a Starting Prefetch Value

There is no single prefetch count that works for every workload. Start with a conservative value and measure actual performance.

For example, if a consumer processes messages that each require a database call, you might begin with a prefetch count of 10.

Then measure:

Messages processed per second.
Processing time per message.
Number of unacknowledged messages.
Queue depth over time.
Consumer memory usage.
Retry and failure rates.

If consumers are frequently idle while the queue remains full, increasing prefetch may help if message delivery or acknowledgement latency is limiting throughput.

If one consumer accumulates many unacknowledged messages while others remain idle, lowering prefetch may improve workload sharing.

Change one setting at a time and compare results under a representative workload.

Step 5: Diagnose Uneven Consumer Processing

When consumers process messages at different rates, inspect both the queue and the consumers.

RabbitMQ management tools can show queue depth, ready messages, unacknowledged messages, consumer counts, and delivery rates.

Pay attention to these patterns:

Observation	Possible explanation	What to investigate
Many ready messages, low delivery rate	Consumers may be unavailable or unable to keep up	Consumer health, connections, CPU, database waits
Many unacknowledged messages	Messages may be assigned but not yet completed	Prefetch, processing duration, acknowledgement logic
One consumer has far more outstanding messages	Deliveries may be unevenly assigned or processing rates may differ	Consumer prefetch, handler concurrency, message sizes
Queue depth grows continuously	Incoming rate exceeds processing capacity	Add capacity, optimize processing, or reduce incoming load
Frequent requeues	A processing failure or retry loop may exist	Error handling, dependencies, dead-letter configuration
Consumers are idle despite a nonempty queue	Consumers may be blocked, paused, or unable to receive more deliveries	Channel state, connection health, acknowledgement behavior

Do not use the unacknowledged message count alone to conclude that a consumer is slow. A consumer with concurrent handlers may legitimately have several messages in progress.

Correlate queue metrics with application-level processing duration and error logs.

Step 6: Understand Prefetch and Consumer Concurrency

Prefetch and application concurrency are related, but they are separate controls.

Suppose you configure a prefetch count of 20 and run one message handler at a time. The consumer can have many outstanding deliveries, but only one handler may be processing a message at any moment.

If your application processes messages concurrently, a larger prefetch count may be needed to keep all workers busy. However, it should be aligned with the number of in-flight operations the application can safely handle.

For example, a consumer that can process five messages concurrently might start with a prefetch count near five or somewhat higher, then adjust based on measured latency and throughput.

This is only a starting point. If processing duration varies widely, a different value may be better.

Do not increase concurrency without checking downstream limits. Too many simultaneous database operations or external API calls can overload dependencies and make the entire system slower.

Step 7: Avoid Losing Messages During Failures

Manual acknowledgements help ensure that a message is not considered successfully processed before the application completes its work.

However, acknowledgements do not automatically guarantee exactly-once business processing.

Consider this sequence:

The consumer updates a database.
The database transaction commits.
The consumer crashes before acknowledging the message.
RabbitMQ redelivers the message.

The database update may happen again when the message is processed a second time.

To handle this situation, make the business operation idempotent. For example, use a stable message ID and a database uniqueness constraint or processed-message record to prevent duplicate effects.

For a long-running operation, make sure acknowledgement timeouts and connection failure behavior are understood for your RabbitMQ version and configuration.

Common Mistakes to Avoid

Setting prefetch extremely high

A high value may increase throughput for some workloads, but it can also cause a consumer to hold more messages than it can process promptly.

Assuming prefetch controls concurrency

Prefetch limits outstanding deliveries. The application determines how many handlers run concurrently.

Using automatic acknowledgements for critical work

Automatic acknowledgement can make delivery simpler, but it is unsuitable when the application needs the broker to track processing completion.

Requeueing every failed message immediately

Immediate requeueing can create repeated delivery loops. Use bounded retries, delays where appropriate, and dead-letter handling.

Sharing a channel without considering concurrency rules

The RabbitMQ .NET client has specific rules for channel operations. Concurrent handlers must not perform unsafe operations on a shared channel.

Ignoring message processing time

A prefetch value that works for short messages may perform poorly when messages take several seconds or minutes to process.

Best Practices for Production Applications

Use manual acknowledgements for work that must complete before delivery is considered successful.
Start with a moderate prefetch count and tune it using real metrics.
Match prefetch to processing concurrency and downstream capacity.
Monitor ready messages and unacknowledged deliveries together.
Keep message handlers idempotent.
Use bounded retries and dead-letter handling for repeated failures.
Follow the concurrency requirements of your RabbitMQ client version.
Measure message-processing latency, throughput, and memory usage.
Test consumer crashes and connection interruptions.
Scale consumer instances when processing demand consistently exceeds capacity.

Summary

RabbitMQ prefetch count affects how many unacknowledged messages can be outstanding for a consumer. A high value may improve throughput for some workloads, but it can also cause consumers to hold large batches of work and make processing less balanced.

Start with a moderate value, inspect queue and consumer metrics, and adjust prefetch based on processing time, concurrency, and downstream capacity.

Most importantly, combine prefetch tuning with manual acknowledgements, controlled retry handling, and idempotent business operations. That approach improves both consumer performance and reliability when messages are delivered more than once.