Creating Azure WebJobs To Parse Messages From Azure Storage Queues

Introduction

 
In this article, we're going to take a look at Web Jobs and help you understand what they are, how they work, and how they can benefit you. We'll get started by taking a look at exactly what a Web Job is. We're going to take a look at utilizing the WebJobs SDK to both simplify the code that we write and extend the reach of our Web Jobs code. Then we'll see what the WebJobs SDK provides in terms of bindings to services, things like Azure Queues, Service Bus, Table and Blob Storage, the other messaging or data-related services. It's going to simplify the job we have of coding and configuring our Web Jobs so that we don't write a lot of the boilerplate code to connect to and access these resources.
 

Road Map

  1. What is an Azure WebJob
  2. Azure WebJob vs Azure Functions
  3. Installing WebJob SDK in Visual Studio
  4. Creating a WebJob using Azure WebJob SDK
  5. Configuring a Queue in Azure Storage to send message
    1. Create a new storage account
    2. Create a queue and send messages
  6. Running the WebJob
  7. Sending message to trigger the Job
  8. Working with Bindings and Triggers
  9. Use Cases

What is an Azure WebJob

 
In their simplest form, Azure Web Jobs allow you to write background code that executes in the context of your web application or website. That means they're not dependent on user interaction, but they are able to access the file system and other things that are available to your website. There's lots of work that needs to happen behind the scenes when a user is not interacting with the site. And that's where Web Jobs come in. A Web Job could run in the background of your application, and they can then access things like your database, the files, either temporary files, log files, source files, for your application, and because it's running in there, you can also have access to the various pieces of your application back end.
 

Azure WebJob vs Azure Functions

 
Azure Functions is a service that you can use to run code in Azure without provisioning any servers or other infrastructure. Because a Function can run on a schedule or continuously, you can use them instead of WebJobs to execute background tasks. You can think of Azure Functions as the logical successor to WebJobs for most workloads. In fact, Azure Functions is built on the WebJobs SDK.
 
For most automated tasks, build Functions because they are more flexible than WebJobs. For example, you can develop and test Functions entirely in the browser and Functions can automatically scale in response to demand.
 
However, code deployed as a WebJob can be developed and maintained alongside the code for the website it is associated with. A website with its WebJobs can be easily packaged, deployed, and managed together, as a single solution. By contrast, Azure Functions are separate from the App Service and must be managed separately.
 

Installing Azure WebJobs SDK

 
Write the following command in the NuGet package manager console in your Visual Studio,
 
Install-Package Microsoft.Azure.WebJobs –Version 2.2.0
 
Creating Azure WebJobs to Parse Messages from Azure Storage Queues
 

Creating a WebJob project

 
Now create a webjob project from Visual Studio as shown. Assume that we are writing a job just to print a message it gets.
  1. Create a new project and choose Azure WebJob

    Creating Azure WebJobs to Parse Messages from Azure Storage Queues

  2. Choose a name for it and press Create

    Creating Azure WebJobs to Parse Messages from Azure Storage Queues
If we look at the Program.cs file we will see a static void Main, and we have this config and the JobHost. These are all part of the WebJobs SDK. That JobHost is going to be responsible for running and blocking inside my console application, and then it's going to be hosting other functions.
 
*The Code below is automatically generated by the project, you don’t have to write it*
 
Creating Azure WebJobs to Parse Messages from Azure Storage Queues 
 
So if we go look at our functions.cs here, notice that we have a specific signature on this method, static void method, and then the parameters, but there's this attribute called QueueTrigger on the message.
 
*The Code below is automatically generated by the project, you don’t have to write it*
 
Creating Azure WebJobs to Parse Messages from Azure Storage Queues 
 
Again, this is part of that WebJobs SDK, and what that says is, go look at a queue, in this case called queue, and when messages arrive, pull them off, pull the string out, and invoke this method, passing in the message. Now let’s create a messaging queue in the azure portal to send messages. But first we will have to create a storage account.
 

Configuring a Queue in Azure Storage to send message

 
For this we will have to create a storage account in the azure portal. 
  • Logon to portal.azure.com
  • In the dashboard click Create a resource

    Creating Azure WebJobs to Parse Messages from Azure Storage Queues

  • Scroll down the list and click on Storage

    Creating Azure WebJobs to Parse Messages from Azure Storage Queues

  • Click on Storage Account

    Creating Azure WebJobs to Parse Messages from Azure Storage Queues

  • Fill in the details

    • Choose subscription
    • Select the resource group or create a new resource
    • Choose the name for storage. It must be unique
    • Choose desired region
    • Leave the other options as default and click Review+Create

      Creating Azure WebJobs to Parse Messages from Azure Storage Queues

  • After the validation is passed click Create

    Creating Azure WebJobs to Parse Messages from Azure Storage Queues

    It will take some time and the resource will be deployed successfully.

  • After the resource is created, go the resource and click on Queues

    Creating Azure WebJobs to Parse Messages from Azure Storage Queues

  • Add a new Queue

    Creating Azure WebJobs to Parse Messages from Azure Storage Queues

  • Add a name and press OK

    Creating Azure WebJobs to Parse Messages from Azure Storage Queues

  • Click on Access Keys and copy the connection string

    Creating Azure WebJobs to Parse Messages from Azure Storage Queues

Running the WebJob

 
Now go back to the WebJob project and make some changes.
  • Go to the app.config file the project directory and paste the connection string in the following,

    • AzureWebJobsDashboard
    • AzureWebJobsStorage

      *The Code below is automatically generated by the project, you don’t have to write it*

      Creating Azure WebJobs to Parse Messages from Azure Storage Queues

  • Change the name of the queue in the cs file with the name of your queue.

    public static void ProcessQueueMessage([QueueTrigger(“messages”)] string message, TextWriter log)

    Creating Azure WebJobs to Parse Messages from Azure Storage Queues

  • Run the app, you will see the following console

    Creating Azure WebJobs to Parse Messages from Azure Storage Queues

Sending Messages to trigger the WebJob

  1. Now go to the portal, navigate to the queues.
  2. Click on Add Message

    Creating Azure WebJobs to Parse Messages from Azure Storage Queues

  3. Type anything a click OK

    Creating Azure WebJobs to Parse Messages from Azure Storage Queues

  4. Now go back to the console and you will see the new Message there.

    Creating Azure WebJobs to Parse Messages from Azure Storage Queues
And that’s how you can use WebJob SDK to receive upcoming messages from the storage queue. Similarly instead of displaying a simple message you can do a heavy background task based on any trigger.
 

Working with Bindings and Triggers

 
We've seen a basic example of just reading a string off the queue and writing it out to the console, but let's take this a step further. I've created a User class here in my WebJobs project that just has a Name, an ID, and a Gender. And what I want to be able to do is get that particular user off the queue.
  • Create a new User class as follows
    1. class User{  
    2.     public int ID {get;set;}  
    3.     public string Name {get;set;}  
    4.     public string Gender {get;set}  
    5. }  
    Creating Azure WebJobs to Parse Messages from Azure Storage Queues

    So if we go back to our Function, we're going to change this from a string message to a User message. And that's going to cause the WebJobs SDK to try to deserialize some JSON that we're going to put on the queue into this class.

  • Change the message from string to User

    *The Code below is automatically generated by the project, you don’t have to write it*

    Creating Azure WebJobs to Parse Messages from Azure Storage Queues

  • Create a Blob as the same way as queue in the AzurePortal

    Creating Azure WebJobs to Parse Messages from Azure Storage Queues

    Creating Azure WebJobs to Parse Messages from Azure Storage Queues

    Now I'm going to add another thing here as an output, and I'm going to put a Blob output on this. Now for the blob we have a variety of different types we can actually use here for the parameter, like any azure storage. This is a binding, a queue is a trigger, which means it will initiate this. This is a binding, meaning that we're going to just have some parameter here, and this binding is going to take care of connecting that up to the resource.

  • Now go to the functions.cs and change the ProcessQueueMessage method as follows,
    1. public static void ProcessQueueuMessage([QueueTrigger(“messages”)] User message, [Blob(“incomingusers/userlog.txt”, FileAccess.Write)] TextWriter bidWriter, TextWriter log){  
    2.     bidWriter.WriteLine($”New User added {message.ID} with Name of {message.Name} Having gender {message.Gender}”);  
    3.     log.WriteLine(message);  
    4. }  
    Creating Azure WebJobs to Parse Messages from Azure Storage Queues

    And now that we have that, if we go back to our portal, remember, we can go to our queue, view that, create a new message.

  • Create a new message as a JSON user object and click OK
    1. {“id”:1 , “name”:”David”, “gender”:”Male”}  
Creating Azure WebJobs to Parse Messages from Azure Storage Queues
 
Now if we go look at our container over here and view the container, you can see there's my userlog.txt.
 
Creating Azure WebJobs to Parse Messages from Azure Storage Queues
 
If I click that, download and open, you will see the data we stored in this.
 
Creating Azure WebJobs to Parse Messages from Azure Storage Queues
 
Creating Azure WebJobs to Parse Messages from Azure Storage Queues
 
If we continue to send messages, then we'd write those to that log file. So it's a very powerful model with these triggers and bindings that allows us to write less code, to take advantage of event-based processing.
 
Similarly instead of Blob storage we can use tables to store our data.
 

Use Cases

  • For running background tasks based on some triggers
  • Can be used with websites unlike Azure Functions
  • Can be used to write data to DB based on an event
  • Can also be used to run continuously in background