Designing Controlled Concurrency in .NET with SemaphoreSlim

Imagine you have 10,000 of orders, and each order need to call a method ProcessOrderAsync to perform some asynchronous operations, such as updating the order in a database and sending messages to the inventory and audit queues. And the codes will look like ths:

foreach (var order in orders)
{
    await ProcessOrderAsync(order, cancellationToken);
}

IsProcessed = true;

Of course, you may not want to start all 10,000 operations simultaneously. Although asynchronous I/O does not require one thread per operation, creating thousands of concurrent operations can still put significant pressure on resources such as database connections, memory, network connections, and downstream services.

Let’s say we want to set the maximum number of asynchronous run to 5, we can manually change the code to become like this:

for (int i = 0; i < orders.Count; i += 5)
{
    var batch = orders.Skip(i).Take(5);

    var tasks = batch.Select(order =>
        ProcessOrderAsync(order, cancellationToken));

    await Task.WhenAll(tasks);
}

This can work perfectly, but we can solve it with another better and elegant way, which is using SemaphoreSlim.

First, we create a method ProcessOrderWithSemaphoreAsync



static async Task ProcessOrderWithSemaphoreAsync(
    Order order,
    SemaphoreSlim semaphore,
    CancellationToken cancellationToken)
{
    await semaphore.WaitAsync(cancellationToken);

    try
    {
        await ProcessOrderAsync(order, cancellationToken);
    }
    finally
    {
        semaphore.Release();
    }
}

The method receive 3 arguments, the order we can want to process, the semaphore that control the concurrency, and the cancellationToken that allow the operation to be cancelled.

This line

await semaphore.WaitAsync(cancellationToken);

means that the operation can continue when a permit is available. In our case, SemaphoreSlim is configured with 5 permits.

If all 5 permits are already being used, the method waits asynchronously until another operation releases a permit.

Although we set the limit to 5 with SemaphoreSlim, it is important to understand that this does not mean the orders are processed in batches of 5.

Think of SemaphoreSlim as a car park with 5 parking spaces. As long as there is an available space, a new car can enter and park. If all 5 spaces are occupied, the next car has to wait until a space becomes available.

This is actually more efficient than our initial approach of processing orders in batches of 5.

Imagine we process orders in batches of 5, and one order takes 10 seconds to complete while the other four orders only take 2 seconds each.

With the batch approach, we cannot start the next batch of 5 until all 5 orders in the current batch have completed. Therefore, even though four orders have already finished after 2 seconds, we still have to wait for the remaining order to finish after 10 seconds.

With SemaphoreSlim, the four orders that finish after 2 seconds immediately free up their parking spaces. Orders 6 to 9 can then start processing while the order that takes 10 seconds continues to occupy the fifth slot.

You can see in 2nd second, order that has finished will free up the space and let other orders process. Order #3 will free up it space after 10 seconds, but other slot already process until order #45. If we run by batch of 5, it only start run order #11 after 10th second.

In other words, SemaphoreSlim maintains a maximum of 5 concurrent operations, but it does not force them to start or finish in groups of 5.

And this is the code of process order asynchronously , where in the finally there is a release code.

try
    {
        await ProcessOrderAsync(order, cancellationToken);
    }
    finally
    {
        semaphore.Release();
    }

The reason we put release in finally is because regardless the order is process successfully or failed in the middle, we need to make sure we free up the space.

Then in our main implementation code, we create something like this:

using var semaphore = new SemaphoreSlim(5);

var tasks = new List<Task>();

foreach (var order in orders)
{
    tasks.Add(ProcessOrderWithSemaphoreAsync(
        order,
        semaphore,
        cancellationToken));
}

await Task.WhenAll(tasks);

IsProcessed = true;

This line

using var semaphore = new SemaphoreSlim(5);

creates a SemaphoreSlim with 5 available permits.

Make sure you have import Threading

using System.Threading;

Next, we create a list to keep track of the tasks:

var tasks = new List<Task>();

Then, we create an asynchronous operation for each order:

foreach (var order in orders)
{
    tasks.Add(ProcessOrderWithSemaphoreAsync(
        order,
        semaphore,
        cancellationToken));
}
...

Notice that we pass the same semaphore instance to every order.

This is important because all orders need to share the same pool of 5 permits.

Finally:

await Task.WhenAll(tasks);

waits asynchronously until all of the order-processing tasks have completed.

SemaphoreSlim provides a simple and effective way to control concurrency in asynchronous .NET applications.

Instead of processing orders one by one or forcing them into fixed batches, we can allow multiple operations to run concurrently while still putting a limit on how many can run at the same time.