A guide to Azure WebJobs - How to implement, deploy and run them

Azure WebJobs have been around for over two years already but I didn’t pay much attention to them until recently when I needed a reliable way to trigger background tasks for a website I’m running. More specifically I needed to build a searchable index based on information available on a bunch of websites. A classic task that should be done periodically on background. Until that I had been running similar tasks using Azure Scheduler but the free version is pretty restricted and I was on a tight budget. So, I decided to take a look at Web Jobs.

What are Azure WebJobs

First of all Azure WebJobs are built into Azure Web Apps (previously Websites). If your site is not an Azure web app then you’re better off using Azure Scheduler or something similar.

WebJobs provide a way to run background tasks either continuously or on schedule. The jobs run on the same virtual machine that your web app is running on and they support a variety of file formats like exe, cmd, bat, sh, php, py and js. Later I’ll show you a way to impelment one as an ASP.NET 5 console application.

Why (or why not) use them

Here are some pros and cons that I’ve found. For me, the pros easily outweight the cons…

Pros

  • Support for continuously running background jobs. If you need to run a CPU intensive task for example whenever a blob is created then there’s really no better way to do it.
  • Support for triggered jobs. You can define your own CRON expression to define your schedule in a very precise way.
  • There is a portal where you can view the execution history and logs. It’s awesome, more about it here.
  • No need to think about authentication because the jobs run on the same VM as your site.
  • You can manage, view and trigger all jobs through Azure portal.
  • There are ready REST API’s to execute and manage the WebJobs.
  • Web jobs can be deployed together with your web application.

Cons

  • Web jobs require your web app to have Always on enabled. Good news though, there is a way around it.
  • Not quite as simple to set up as Azure Scheduler for example.

Creating a simple triggered web job

We’ll be creating the triggered web job as an ASP.NET 5 console application but you can choose a classic ASP.NET 4.5 console app too if you want to. In Visual Studio 2015 add a new project of type Console Application (Package) to your solution. There is nothing special to be aware of here, just make the console app to do whatever you want your web job to do. After all Azure just runs the .exe or .cmd -file that you tell it to. If you wish to use some of the functionality provided by Azure WebJobs SDK you can read more about it here.

Deploying the web job

Before we can deploy it we need to publish the webjob to a local folder. To do this we create a simple Power Shell file named build-webjob.ps1 in our solution folder:

param($webjob, $out)
# example build-webjob.ps1 -webjob MyWebJobsProjectFolder -out output

# activate runtime
& $env:USERPROFILE\.dnx\bin\dnvm upgrade -r clr -arch x86

# web jobs
& dnu publish $webjob --runtime active --no-source --out $out

Now, run that Power Shell script for the console application and zip the output folder. Next login to Azure portal, open settings for your web app and select WebJobs. Click add to create a new web job and provide the ZIP file when asked. Your webjob is now ready to be run which, unfortunately, is still only possible from the Classic portal. The classic portal also enables you to define a schedule but I’ll show you later how to do it with a Cron expression file instead.

Azure WebJobs in portal

Deploying a WebJob together with a Web App

Deploying the webjob together with your web app is pretty easy. All you need to do is copy the output directory created by the Power Shell script to a specific folder (this is ASP.NET 5 specific, in ASP.NET 4.5 the App_Data folder should be under your web project’s root folder):

[solution_dir]/src/[webproject_dir]/wwwroot/App_Data/jobs/triggered

Now, when you publish your web app the App_Data directory will also be deployed containing your webjob! If you want, you can easily automate the publishing of the web job to the correct directory in your CI or build process.

Running the job from Azure portal

As said, the new Azure portal does not yet allow to manually run web jobs but I hope it will be added soon. So switch to the classic portal and select Web Jobs under your web app. You’ll see your web jobs listed together with some useful information and a button to run a job manually. We however, want to specify a schedule so that the job executes automatically at specific times.

Defining a schedule using CRON expressions

Luckily you can define the schedule for a web job with a cron expression. You define the schedule by creating a settings.job file under the web job’s root folder (i.e. the output folder of the Power Shell script above).

{"schedule": "0 0,20,40 * * * *"}

This cron expression that I use defines a schedule where the job is set to run every even hour as well as 20 and 40 minutes past it (i.e. three times every hour). You’ll find lots of examples in the internet to build a cron expression and schedule to fit your needs.

Now when you deploy the web job (and the settings.job file), Azure will pick up the schedule and execute your web job according to it.

The Always On -requirement and how to get around it

There’s one more thing you should be aware of or otherwise you’ll soon be wondering why your web job isn’t following the schedule you’ve defined. To keep Azure web jobs running around the clock your web app needs to be always on. You can just enable this in the portal under Application settings unless you’re running a free or shared web app. There’s a significant price increase when switching from a shared to a basic web app but luckily there’s a way around it.

The solution is… Ping the related Kudu Service site often enough to keep your web app always on. This can be easily done using a service like Pingdom or HappyApps.io (free). What you need to do is make a periodical ping (e.g. every 5 minutes) to https://yourwebapp.scm.azurewebsites.net/ using basic authentication with your deployment credentials. This simple trick will prevent your web app from ever being unloaded after idle time and the web job will keep executing perfectly.

Viewing execution history and logs

If you didn’t know it yet, there’s a Kudu Service hosted and running for every Azure web app. It lives at https://yourwebapp.scm.azurewebsites.net so go ahead and take a look. It even has a nice user interface for viewing the status of web jobs, execution history and logs. Navigate to https://yourwebapp.scm.azurewebsites.net/azurejobs to view these. All output from your console application (e.g. Console.WriteLine('Output from a WebJob')) can be seen here under each run - cool huh!

List of web job runs in Kudu

Details of a web job run in Kudu

Best of all, there’s even a REST Api for Kudu that allows you to access and manipulate a lot of Kudu’s data, including all the information related to Web Jobs.

Have fun and happy scheduling!