SQS Message Polling using C#

Introduction

Amazon Simple Queue Service (SQS) is a fully managed message queuing service that allows users to exchange and store messages between services. The service places messages into a queue. Users or services then retrieve these messages, and after processing, the messages are deleted from the queue.

The benefits of using SQS are,

Availability
Reliability
Scalability

In this article, we are going to learn about how to poll messages from SQS using C#.

Types of SQS Queue

Two types of SQS queues are available.

Standard queues
FIFO queues

Standard Queue

Standard queues are optimized for high throughput, permitting an unlimited number of transactions per second and ensuring at least one message delivery. However, strict message ordering is not guaranteed with standard queues.
Standard queues are used in scenarios where speed and scalability take precedence over sequence, such as in large-scale data processing or handling background tasks.

FIFO Queue

FIFO queues guarantee that messages are delivered in the precise order they are sent, providing exact-once processing.
FIFO queues are used for workflows where maintaining order accuracy is important, such as bank transactions, order management systems etc.

SQS Message Polling

The ReceiveMessageRequest is a request used to receive messages from a queue. The receiveMessageRequest method takes the below arguments.

QueueUrl (Mandatory)
MaxNumberOfMessages (Optional)
VisibilityTimeout (Optional)
WaitTimeSeconds (Optional)
AttributeNames (Optional)

Once a message is successfully processed, it should be deleted from the SQS. To delete the message from the queue, we need to call the delete message method (DeleteMessageAsync) explicitly. The QueueUrl and the messageReceiptHandle are inputted using the DeleteMessageAsync method. The ReceiptHandle is a unique identifier associated with receiving a message. Each time you receive a message (even if the same queue message is received again), the ReceiptHandle will be different.

private IAmazonSQS _sqsClient = new AmazonSQSClient(Amazon.RegionEndpoint.EUWest2);

public async Task FunctionHandlerPollMsg(ILambdaContext context)
{
    var messageResponse = await GetMessage();
    foreach (var message in messageResponse.Messages)
    {
        try
        {
            Console.WriteLine(message.Body);
            await DeleteMessage(message);
        }
        catch (Exception ex)
        {
            context.Logger.LogError(ex.Message);
        }
    }
}

public async Task<ReceiveMessageResponse> GetMessage()
{
    var request = new ReceiveMessageRequest
    {
        QueueUrl = _queueUrl
    };

    return await _sqsClient.ReceiveMessageAsync(request);
}

public async Task DeleteMessage(Message message)
{
    var request = new DeleteMessageRequest
    {
        QueueUrl = _queueUrl,
        ReceiptHandle = message.ReceiptHandle
    };

    var response = await _sqsClient.DeleteMessageAsync(request);

    if (response.HttpStatusCode != System.Net.HttpStatusCode.OK)
    {
        Console.WriteLine($"({message.ReceiptHandle}) - Error processing delete message request");
    }
}

The following properties are important when reading the message.

MaxNumberOfMessages
WaitTimeSeconds
VisibilityTimeout

MaxNumberOfMessages

The maximum number of messages to return from SQS. Amazon SQS never returns more messages than this value. A value between 1 and 10 with the default of 1.

var request = new ReceiveMessageRequest
{
    QueueUrl = _queueUrl,
    MaxNumberOfMessages = 5
};

Amazon SQS returns a number of messages equal to the count specified in MaxNumberOfMessages. If there are not enough messages in the queue, it can return fewer messages.

WaitTimeSeconds

The WaitTimeSeconds property is used to add explicit delay to the message polling. The duration (seconds) for which the request should wait before returning a response. If the messages are available, the request returns sooner than the WaitTimeSeconds value. If no messages are available and the wait time expires, the request returns an empty response.

var request = new ReceiveMessageRequest
{
    QueueUrl = _queueUrl,
    WaitTimeSeconds = 10
};

WaitTimeSeconds property value between 0 and 20 with the default of 0. This WaitTimeSeconds property is used to enable Short or Long polling in SQS.

Short Polling

The WaitTimeSeconds value 0 is referred to as Short Polling
Immediately returns a response, even if the queue is empty.
Ideal for applications where immediate responsiveness is important.
It can be more costly due to frequent requests that may return no messages.

Long Polling

When the WaitTimeSeconds value is greater than 0, it’s called Long Polling.
Waits until a message is available in the queue or until the timeout period
Suitable for reducing costs in applications where immediate response time is not critical.
More cost-effective as it reduces the number of empty responses and server calls.

VisibilityTimeout

When a message is received from a queue, it becomes temporarily invisible to other consumers for the period. The VisibilityTimeout property is used to set the time for invisible to other consumers.

If not specified, the default visibility timeout for the queue is used, which is 30 seconds. It takes a value (in seconds) between 0 and 12 hours. If the consumer fails to process the message and deletes the message, the message will be available in the queue again after 30 seconds.

var request = new ReceiveMessageRequest
{
    QueueUrl = _queueUrl,
    VisibilityTimeout = 20
};

Once a message is successfully processed, it should be explicitly deleted from the SQS by calling the DeleteMessageAsync method. If the application fails to process/delete the message within this timeframe, the message will become visible again in the queue. It will be processed multiple times and sent to the DLQ if the maxReceiveCount is exceeded.

Example

var request = new ReceiveMessageRequest()
{
    QueueUrl = _queueUrl,          // The URL of the queue
    MaxNumberOfMessages = 1,       // Get 1 message
    VisibilityTimeout = 30,        // Hide it for 30 seconds
    WaitTimeSeconds = 20           // Wait up to 20 seconds for a message
};

Conclusion

In this article, you have learned the following topics,

What is SQS and its types
How to poll messages from SQS