Azure Webjobs part 1 – using Azure Storage Message Queues
Azure WebJobs are pretty amazing! They’re similar to Azure Worker Processes but with a couple of differences. First they can be triggered by an action – this action is usually the addition of a message to an Azure Message Storage table. Second, they run on an Azure Website (recently renamed to “Web Apps”) and as a result you can run a number of WebJobs entirely for free!
(Oh – if you don’t know what a Worker Processes is, they’re Azure’s answer to Scheduled Tasks. If you don’t know what a Scheduled Task is, you probably clicked the link to this blog post by mistake. Maybe you were looking for more info about the colour Azure, in which case – here you go – you’re welcome)
I’ve seen a couple of people I follow on Twitter talk about how awesome Azure WebJobs are recently. For example, Troy Hunt uses WebJobs for his pretty awesome and super-scaled “Have I Been Pwned” website:
Blogged: Azure WebJobs are awesome and you should start using them right now! http://t.co/bDAUSR5Ubu
— Troy Hunt (@troyhunt) January 28, 2015
And Karoline Klever recently discovered how awesome they are on one of her projects:
Creating a custom log appender that pushes messages to an #Azure Queue and a WebJob that processes them. So simple!
— Karoline Klever (@karolikl) April 7, 2015
So I’ve been looking for the right opportunity to try these out more for myself and a few weeks ago I was able to!
As I said at the beginning of this post, WebJobs can be triggered by adding data to an Azure Message Storage table. This post will focus on creating a Message Storage table and adding a message to it. In part 2 of this “not yet an epic” blog post series I will show you how to create the actual WebJob and do something when your data is queued up in the Message Storage table.
First things first
If you haven’t used Azure before, you’re probably going to need to install the Azure SDK so you can access all of this goodness.
You may also need to install the Azure Storage NuGet package if it’s not installed already locally.
Set up a storage account
So if you haven’t used Azure Storage before, the first thin you will need to do is create a Storage account through the Azure Management Portal. Azure Storage is billed based on the size of what you’re storing, as well as the number of transactions in and out of the storage. The costs are pretty tiny so for a quick example such as this one you probably won’t even be charged! However if you’re feeling particularly tight-fisted or don’t have access to a credit card at the moment, you can use your local Azure emulator instead (this should have been installed along with the Azure SDK from the previous step).
If you do create a storage account in Azure, you will see from the Azure portal that your storage account has 3 services – Blobs, Tables and Queues. Blobs are used to store files, Tables are used for data storage, and Queues are used for storing messages. The service we will be using is the Queues service.
Before you write the code to add messages to the Queue storage you need to know two things – the connection string for your storage account and the name of the queue you’re adding messages to.
To find the connection string for your storage account, view your storage account dashboard in the Azure Management portal, and click the ‘Manage Access Keys’ link at the bottom of the page. You should see a box that looks like this:
The Connection String for your Azure storage will looks like this:
DefaultEndpointsProtocol=https;AccountName=[Storage Account Name];AccountKey=[Primary Access Key]
You can store this in your application in 2 ways – either add it as an AppSetting value in your web.config like this:
Or if you’re using an Azure Cloud Service for your site you can add it to the settings of the Cloud Service Role like this:
Message Queue Name
You can actually define this in your code, and if the queue with that name doesn’t exist, you can get your code to create it. We can see that in action when we look at the code in the next bit. WARNING – your message queue name needs to be all in lower case otherwise you (like me) will spend hours and hours trying to figure out why it isn’t working!!!
Don’t bore us, get to the chorus!
Ok so now we have the boring config stuff out of the way, the interesting code bit is refreshingly straight forward – you can add data to your Message Queue in just 6 lines of code!!!
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting(“StorageConnectionString”));
// Next, create a queue client
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
// Then retrieve a reference to a queue – here’s where you give your queue a name
CloudQueue queue = queueClient.GetQueueReference(“mymediaqueue”);
// Create the queue if it doesn’t already exist
// Penultimate step – create a message and add it to the queue.
CloudQueueMessage message = new CloudQueueMessage(“this is the data I want to add to my message queue”);
// Finally, add your message to the queue
That’s it – pretty simple eh?
In reality though, you probably want to store a message that is a bit more useful than the example above! The good news is that because the message queue stores a string value, you can store serialized objects in there, and they will be deserialized to objects by your WebJob code in the next step.
So you could do something like this:
Id = 1,
SiteUrl = “http://www.mysiteurl.com”
CloudQueueMessage message = new CloudQueueMessage(JsonConvert.SerializeObject(messageQueueData));
A quick final point to make – you can view the contents of your message queue at any time through Visual Studio’s Server Explorer. Just connect to your Microsoft Azure account and navigate through to your storage account. You will see the Queues service there and the queue that you just created. Double click on your queue and you should see the message you just added!