IronPython Web Roles for Windows Azure

Following up to my previous post on IronPython worker roles, I'm going to discuss how to implement web roles using IronPython. The code discussed here is on bitbucket, as usual.

There are currently two ways to implement IronPython web roles on Azure: using the ASP.NET Dynamic Language Support or using NWSGI. Someday I hope the IronRubyMVC project matures to be an option as well.

Using ASP.NET Dynamic Language Support

This example can be found in the PyWebRole folder of the project.

First, you'll need to download the ASP.NET Dynamic Language Support (ASP.NET DLS) files. Next, create a standard C# web role and delete all of the .cs files except WebRole.cs. After that, there's really not much to it – drop the assemblies from the ASP.NET DLS package into the project's bin folder and add the following bits to the web.config file (alternatively, just copy the Web.config from the project):

<configSections>
    <!-- Add this to the existing <configSections> element -->
    <section name="microsoft.scripting" type="Microsoft.Scripting.Hosting.Configuration.Section, Microsoft.Scripting, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" requirePermission="false"/>
</configSections>

<system.web>
    <!-- Replace the existing <pages> element (if any) -->
    <pages compilationMode="Auto" pageParserFilterType="Microsoft.Web.Scripting.UI.NoCompileCodePageParserFilter" pageBaseType="Microsoft.Web.Scripting.UI.ScriptPage" userControlBaseType="Microsoft.Web.Scripting.UI.ScriptUserControl">
        <controls>
            <add tagPrefix="asp" namespace="System.Web.UI" assembly="System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"/>
            <add tagPrefix="asp" namespace="System.Web.UI.WebControls" assembly="System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"/>
        </controls>
    </pages>
</system.web>

<system.webserver>
    <modules>
        <!-- Add this to the existing <modules> element -->
        <add name="DynamicLanguageHttpModule" preCondition="integratedMode" type="Microsoft.Web.Scripting.DynamicLanguageHttpModule"/>
    </modules>
</system.webserver>

<!-- Add this whole section -->
<microsoft.scripting debugMode="true">
    <languages>
        <language names="IronPython;Python;py" extensions=".py" displayName="IronPython 2.6" type="IronPython.Runtime.PythonContext, IronPython, Version=2.6.10920.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
    </languages>
</microsoft.scripting>

From here, just follow the directions on the ASP.NET DLS page and in the package to create an ASP.NET Dynamic Language site.

NWSGI

First, download NWSGI 2.0. Next, deploying NWSGI to Azure is exactly the same as deploying it to any other server using xcopy. That's about it. You can see this example in the PyHelloWorld folder of the project.

Now What?

Unfortunately, now you're pretty much on your own. So far, none of the big Python web frameworks work 100% on IronPython, there is no Python library to access Azure's storage tables, blobs, or queues, and the .NET storage client library relies on LINQ support (which IronPython doesn't have, yet).

Personally, I plan to implement the data access in C# and call that from IronPython, running my own web framework. We'll see how it goes.

IronPython Worker Roles for Windows Azure

Curiosity has finally got the better of me and I've started looking into Windows Azure again. It's matured quite a bit since I looked at it last year and now looks like a pretty solid platform to work with.

Aside from using NWSGI to write a web role (which I'll show later), I wanted to see if it was possible to write a worker role in Python. Happily, it is, and it's not that complicated. In fact, it's pretty similar to how NWSGI works – load up a Python file and run some functions.

Worker Role Requirements

A standard C# worker roles requires three functions: OnStart, OnStop, and Run:

public class PyWorkerRole : RoleEntryPoint
{
    public override bool OnStart() { /* ... */ }
    public override void OnStop() { /* ... */ }
    public override void Run() { /* ... */ }
}

This can be mapped to a Python module in a fairly straightforward fashion:

def start():
    return True

def run():
    pass

def stop():
    pass

This has some advantages and disadvantages compared to using a class, but I like it for its simplicity.

The Implementation

Azure requires an actual .NET class to implement a worker role, so we create one that hosts the IronPython engine. This is a good example of how to embed IronPython to run very simple scripts. The core IronPython hosting function is shown here; for the rest, see the files linked below.

private void InitScripting(string scriptName)
{
    this.engine = Python.CreateEngine();
    this.engine.Runtime.LoadAssembly(typeof(string).Assembly);
    this.engine.Runtime.LoadAssembly(typeof(DiagnosticMonitor).Assembly);
    this.engine.Runtime.LoadAssembly(typeof(RoleEnvironment).Assembly);
    this.engine.Runtime.LoadAssembly(typeof(Microsoft.WindowsAzure.CloudStorageAccount).Assembly);                 
    
    this.scope = this.engine.CreateScope();
    engine.CreateScriptSourceFromFile(scriptName).Execute(scope);             
    
    if(scope.ContainsVariable("start"))
        this.start = scope.GetVariable<Func<bool>>("start");
    
    this.run = scope.GetVariable<Action>("run");
    
    if(scope.ContainsVariable("stop"))
        this.stop = scope.GetVariable<Action>("stop");
}

First, we create a ScriptEngine and add some useful assemblies;  then we create a Scope to execute in; then we actually execute the script. Finally, we try to pull out the functions and convert them to C# delegates; run is required but start and stop are optional. Those delegates are called from the C# wrapper (from Run, OnStart, and OnStop, as appropriate).

The rest of the file is pretty much taken from the worker role template, so I'll leave it out.

Doing Actual Work

Now, a worker role needs some actual work to do – usually, reading items from a queue and processing them. Happily, the Azure StorageClient library is perfectly usable from IronPython.

from Microsoft.WindowsAzure import CloudStorageAccount
from Microsoft.WindowsAzure.StorageClient import CloudQueueMessage, CloudStorageAccountStorageClientExtensions

def run():
    account = CloudStorageAccount.FromConfigurationSetting("DataConnectionString")
    queueClient = CloudStorageAccountStorageClientExtensions.CreateCloudQueueClient(account)
    queue = queueClient.GetQueueReference("messagequeue")

    while True:
        Thread.Sleep(10000)

        if queue.Exists():
            msg = queue.GetMessage()
            if msg:
                Trace.TraceInformation("Message '%s' processed." % msg.AsString)
                queue.DeleteMessage(msg)

The only catch is that CreateCloudQueueClient is an extension method, so it must be called as a static method on the CloudStorageAccountStorageClientExtensions class.

Using the Code

To actually use the code, create a C# worker role as per usual, but replace the generated class file with PythonWorkerRole.cs (see below). Next, add the IronPython assemblies as references to the project. Then, create a string setting for the role (under the Cloud project's Roles folder) called ScriptName and set it to the name of the script file. Finally, add a .py file to the worker role and ensure that (under 'Properties') its 'Build Action' is 'Content' and 'Copy to Output' is 'Copy if Newer'.

The code can be downloaded from my PyAzureExamples repository, including zip archives of it. It includes the PyWorkerRole project and the Cloud Service project.

NWSGI on Window Azure

I have had a couple of people ask about NWSGI on Windows Azure. Architecturally (as far as NWSGI is concerned), Azure is just IIS7 running in medium trust. You have to use xcopy deployment, which makes me glad I went to the effort of supporting it.

NWSGI 0.6 actually supports medium trust. I only discovered after the release that the error I was hitting was a bug in Cassini; it works fine in medium trust on IIS7. With that hurdle out of the way, there's no reason NWSGI shouldn't work on Azure.

And, what do you know, it works just fine: NWSGI "Hello, World" for Windows Azure. You'll need the Azure SDK and the Visual Studio tools to run it.

The big downside is that every NWSGI app would have to include its entire framework and the Python standard library (or whatever subset that it uses).