Implement Azure Queue Using ASP.NET Core Console Application

Introduction

Today, in this article, we will discuss how to develop a web application or console-based application to insert a message into the Azure Queue Storage. Now, as we all know Queue Storage is a part of the Azure Storage. So, first, we will discuss some basic concepts about Azure queue storage and then we will discuss how to store data into the Azure queue storage.

As a developer, we all know about queues. Most of us have already used or written a similar type of data structure at least once in our development-related work phase. In general, it is a collection that always has two primary methods: enqueue (a method to insert new records to the queue at the end) and dequeue (to get the topmost record from the queue). Yes, your assumption is correct. Microsoft Azure Queues are normal queues as we understand them. However, it is more than a class or a concept compared to others. Microsoft Azure Queues are a ready-to-use service that loosely connects components or applications through the Azure platform infrastructure.

Azure queue storage

Azure Queue Storage is a type of message queuing service provided by the Azure Platform which provides the queue storage infrastructure for a REST-based interface within and between the different applications and services. With the help of Azure Queue storage service, we can store a large number of messages that can be accessed from anywhere via authenticated calls using HTTP or HTTPS. So, in simple words, Azure queues are queues located in the Cloud platform and we can use it for exchanging messages between different components either in the cloud or on-premise. Every message typically represents a task created by someone (“produced”) which has to be processed by someone else (“consumer”). Every message always has a small body and some attributes like time-to-live, which we can use to configure our service. As Microsoft Azure always ensures that any dequeued message will be invisible to the other consumer or listener, we could imagine many producers and consumers as well as a one-to-one relationship scenario. So, the main benefit of the Azure Queue Service is the loose coupling.

As discussed in the above section, Azure Queues are a RESTful service through which we can use to enqueue and dequeue messages as operate (create, delete) our queues. For performing this type of work programmatically, Microsoft Azure provides many language-specific wrappers APIs (like .NET, Node.js, Java, PHP, Ruby, Python, etc) through which we can develop applications to send or receive REST call directly to access the Azure Queue storage.

Advantages and disadvantages of Azure queue storage

The main key advantages of the Azure Queue Storages are

• Azure Queue Storage service is much cheaper compared to the others. Azure Queues are charged based on our pay-per-use policy depending on the desired redundancy level, the required storage space, and several transactions (i.e. read, write, delete).
• Queue Storage data is much more secure since to access the data we need to use either HTTP or HTTPS protocol through applications
• We just need to pay only for the storage and operations. There is no long-running cost such as an Event Hub or a service bus.

Despite the above advantages, Azure Queue storage also has some disadvantages, like

• There is no provision or option to mention any order sequence of the message, so messages can be received the Queue service randomly from different producers.
• There is no subscription system in the Azure Queue service. So, to know whether new messages have arrived or not, we need to perform a pull to delete the new messages.
• The maximum size of every message is 64 KB.
• By default, the maximum TTL (Time-To-Live) for each message is 7 days.

An Azure Storage Queue will work just as we expect. We normally use a library that contains an Abstract API, through which we will do REST calls to a storage layer where messages are stored. The main difference between the Blob Storage and the Queue Storage is that the Queue Storage system has a retention time or TTL of 7 days and a size limit of 64 KB.

Azure Queue - Where or When to Use? A Typical Scenario

As per the discussion, it is clear that the main reason behind the use of Azure Queue is to separate components and loosely connect them. Let's imagine two different components of our application trying to exchange data, maybe one of them on-premise component (say A) and another in the cloud environment (say B). Now, when we use common exchange mechanisms such as calling a web service from A to B (or vice-versa), we always face some disadvantages related to this.

1. We need to rely on or trust that both components will be available at the same time or simultaneously. If one of them is down, then there is no communication.
2. We need to implement try or retry logic to get things Up and running again after an outage
3. It is very hard to scale up if more workload is present.

In Azure Queue Service, we have a third player that connects the two components and acts as both a buffer and a mediator. It means if the consumer partner is down, then still insert messages in the queue process while it's waiting for the other component to back online. Also, the Azure Queue service is very easy to scale up. What we need to do for scaleup: just add more consumers (i.e. Worker Roles) and our queues will be processed in parallel. In Azure Queue Storage, Scaling up and down is always be a dynamic process – since we can add or remove workers automatically depending on the actual size of the Queue, hence it will be fast on heavy load and save costs when the queue is empty.

Perquisites required

1. Microsoft Visual Studio 2017
2. The account is in Azure Portal.

If you don’t have an existing Azure account, then you can create a free trial account in the Azure portal using your email ID.

How to create a message in the Azure queue storage account using the Azure portal

To use Azure Queue Storage, we first need to create a Storage Account in Azure Portal. Once the Storage Account has been created, click on the account to open it in Storage Account Explorer.

Step 1. Now Click on the Queues option in the Storage Explorer.

Step 2. Under the Queues option, click on the Queue button to create the queue options.

Step 3. In the Add Queue section, provide the unique Queue Name and click on the OK button.

Step 4. Now in the Queue Explorer window, click on the Add Message button to add a new Message under the selected Queue.

Step 5. In the Add Message area, provide the Message in the Message Text Box and click on the OK button. As shown in the picture, the default TTL for the Message is 7 days.

Step 6. Now, the newly created message is displayed in the Queue Explorer.

Step 7. If we want to Dequeue the Message, then we need to click on the Dequeue message button and then click on the Yes Button.

Step 8. The Dequeue Button will delete the first element of the message list under the queue.

Upload Files in Azure Queue Share using the .NET Core Console Application

Step 1. Now open Microsoft Visual Studio 2017 and click on File --> New --> Projects.

Step 2. Select the Console App Project template and Click on the OK Button

Step 3. Before going to start coding, we first need to install some packages that help us to implement Azure Queue Storage in the console applications.

Step 4. In the Solution Explorer, right-click on our project and choose Manage Nuget Packages.

Step 5. Now, In the Nuget Package Manager, select Browse.

Step 6. Now, Search for the Package Microsoft.Azure.Storage.Queue and then Install.

Step 7. Similarly, install the below packages.

Microsoft.Azure.Storage.Common

Step 8. Now write down the Below code to create a new message programmatically in the Azure Queue Storage.

public static string connstring = "DefaultEndpointsProtocol=https;AccountName=accountName;AccountKey=keyName;EndpointSuffix=core.windows.net";
static void Main(string[] args)
{
    AddMessage();
    Console.ReadKey();
}
public static void AddMessage()
{
    CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connstring);
    CloudQueueClient cloudQueueClient = storageAccount.CreateCloudQueueClient();
    CloudQueue cloudQueue = cloudQueueClient.GetQueueReference("messagequeue");
    CloudQueueMessage queueMessage = new CloudQueueMessage("Hello, Message Created by Console Application");
    cloudQueue.AddMessage(queueMessage);
    Console.WriteLine("Message sent");
}

Step 9. Run the program and then check the Azure Portal for the new Message.

Step 10. Now, we want to read the message by using the Application. For that purpose, add the below method.

public static string connstring = "DefaultEndpointsProtocol=https;AccountName=accountName;AccountKey=keyName;EndpointSuffix=core.windows.net";
static void Main(string[] args)
{
    RetrieveMessage();
    Console.ReadKey();
}
public static void RetrieveMessage()
{
    CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connstring);
    CloudQueueClient cloudQueueClient = storageAccount.CreateCloudQueueClient();
    CloudQueue cloudQueue = cloudQueueClient.GetQueueReference("messagequeue");
    CloudQueueMessage queueMessage = cloudQueue.GetMessage();
    Console.WriteLine(queueMessage.AsString);
    cloudQueue.DeleteMessage(queueMessage);
}

Step 11. Now, run the above program and check the output. It will display the message that we created first.

Step 12. Now, again go to the Azure Portal and click on the refresh button.

Step 13. In step 9, it shows the Dequeue count for the message is 0 after the message is created, so we do not consume that message anymore. But now, we consume the message through the Application. So that’s why the Dequeue Count increased to 1 for the first message. If we run the program again, then it will show us the next message (which we created with the application) and make the dequeue count 1 for that message.

Conclusion

In this article, we discussed how to store data using the ASP.NET Core application in Azure Queue Storage. Any suggestions feedback or queries related to this article are most welcome.