NWSGI 2.0: Advanced Dispatching

Function Callables

Let’s say we’re dealing with a much larger Python application, coolapp, that is installed at C:\coolapp-1.1\. Unfortunately, coolapp doesn’t provide a ready-made .wsgi file for us, but it does provide a function(coolapp.dispatchers.wsgi_application) that we can use by writing a little wrapper that looks something like:

import sys
sys.path.append("C:\coolapp-1.1")
import coolapp.dispatchers.wsgi_application
application = coolapp.dispatchers.wsgi_application

We could save this as coolapp.wsgi and follow either of two methods from last time, but since we’re not doing anything fancy in the wrapper, NWSGI provides a shortcut:

<wsgi>
    <pythonPaths>
        <path path="C:\coolapp-1.1" />
    </pythonPaths>
    <scriptMappings>
        <scriptMapping scriptName="coolapp.wsgi" callable="coolapp.dispatchers.wsgi_application" />
    </scriptMappings>
</wsgi>

First, we add C:\coolapp-1.1 to Python’s path so that we can import it. Next, we tell it that any requests for coolapp.wsgi should be dispatched to the function coolapp.dispatchers.wsgi_application. This is completely equivalent to the wrapper file, but saves us from creating it. Visiting http://example.com/coolapp.wsgi/ will run coolapp.

Class Callables

Functions aren’t the only things in Python that are callable; classes are also callable (it creates an instance of the class). Let’s say we upgrade coolapp to version 2.0 (installed in C:\coolapp-2.0). The team has made some changes, and their WSGI application is now a class, WsgiApplication. This class has a __call__ method (making its instances callable as well!), so our WSGI application is an instance of the WsgiApplication class. We could use a coolapp.wsgi wrapper again:

import sys
sys.path.append("C:\coolapp-2.1")
import coolapp.dispatchers.WsgiApplication
application = coolapp.dispatchers.WsgiApplication()

Again, our little wrapper doesn’t do much, so NWSGI provides a shortcut:

<wsgi>
    <pythonPaths>
        <path path="C:\coolapp-2.0" />
    </pythonPaths>
    <scriptMappings>
        <scriptMapping scriptName="coolapp.wsgi" callable="coolapp.dispatchers.WsgiApplication()" />
    </scriptMappings>
</wsgi>

This time, NWSGI will create an instance of WsgiApplication, and then call the instance and return the result to IIS. This shortcut will only work if the class doesn’t require any arguments; if it does, it needs to be wrapped in a .wsgi file (when in doubt, you can always use a wrapper file). Again, visit http://example.com/coolapp.wsgi/ to run the application.

There’s actually one more case: Python classes that implement the iterator protocol, but I don’t think two many applications are implemented that, so I’ll skip it. PEP 333 gives an example if you’re really interested.

NWSGI 2.0: Dispatching

NWSGI has a fairly simple task: given a URL, call some Python code that produces some output. The devil, as always, is in the details.

What is WSGI?

NWSGI is an implementation of the Python WSGI specification (PEP 333). WSGI is the Web Server Gateway Interface, “a simple and universal interface between web servers and web applications or frameworks”, as defined in PEP 333.

OK, so what does that mean?

The purpose of WSGI is to define how web servers (i.e. Apache, IIS, etc.) talk to Python web application or frameworks (i.e. Trac, Django, etc.). In theory, a web server with a WSGI implementation (such as mod_wsgi for Apache, or NWSGI for IIS) should be able to run any web application that is implemented as a WSGI application (prior to WSGI, many applications/frameworks were server-specific). Of course, that doesn’t always pan out in practice, but it’s a start.

What is a WSGI application?

A WSGI application is surprisingly simple:

def simple_app(environ, start_response):
    """Simplest possible application object"""
    status = '200 OK'
    response_headers = [('Content-type','text/plain')]
    start_response(status, response_headers)
    return ['Hello world!\n']

The simple_app function is the entire application! A WSGI application is a callable (a function, mostly) that takes two arguments and returns an iterable (a list or generator, mostly). There’s some extra goo in there as well (start_response, for example) that’s not really relevant as far as dispatching is concerned. Writing WSGI applications is probably a book’s worth of content, so I’ll leave it at that for now.

Finding Callables in .wsgi Files

Local Files

Now that we have a callable for our application, NWSGI needs to know how to find it. The simplest way is to simply drop the file (let’s call it simple.wsgi) into the application root (i.e. C:\inetpub\wwwroot\, next to web.config). If you make a request to http://example.com/simple.wsgi/, IIS will see that .wsgi is associated with NWSGI and pass the request to it. NWSGI will then open simple.wsgi, try to find a variable called application (which is, by convention, the name of WSGI callables), call it, and return the result to IIS.

In this case, however, our callable is named simple_app, not application. To be able to run our app, NWSGI needs to know this! To do this, there needs to be a script mapping telling NWSGI how to run simple.wsgi.

<wsgi>
    <scriptMappings>
        <scriptMapping scriptName="simple.wsgi" callable="simple_app" />
    </scriptMappings>
</wsgi>

This configuration tells NWSGI that for the script simple.wsgi, it should use simple_app as the callable instead of application. (NOTE: This exact syntax requires 2.0b2 or later).

Other Files

Best practices for WSGI applications say that you should not put your application files in the web-exposed directory. This means that simple.wsgi should really live somewhere else; let’s say C:\simple\simple.wsgi. Of course, now NWSGI doesn’t have the slightest clue where it is, so we have to tell it:

<wsgi>
    <scriptMappings>
        <scriptMapping scriptName="simple.wsgi" physicalPath="C:\simple\simple.wsgi" callable="simple_app" />
    </scriptMappings>
</wsgi>

There is no longer a .wsgi in our application folder, but IIS doesn’t care. If you visit http://example.com/simple.wsgi/ with this configuration, IIS still happily passes the simple.wsgi script onto NWSGI. NWSGI looks at the script mappings and finds that simple.wsgi maps to C:\simple\simple.wsgi, so it loads that file instead (without looking for simple.wsgi in the application directory), looks up simple_app, and calls it.

Using .wsgi files is the simplest way to deploy an application, but NWSGI has a couple of shortcuts to make things easier for certain applications.

NWSGI 2.0: Configuration Details

For the most part, NWSGI can be used without any configuration. All you have to do is create a .wsgi file, throw it in a directory, let IIS know what’s going on, and you’re good to go! Of course, if it was always that easy, configuration wouldn’t exist. To really work with NWSGI, you need to understand its configuration. It helps if you’re already familiar with web.config files; if not, check this out (but hurry back!).

First Things First

NWSGI uses .NET’s built-in configuration system (System.Configuration), so the first thing that needs to be done is to let the .NET Framework know about the custom NWSGI configuration section:

<configSections>
    <section name="wsgi" type="NWSGI.WsgiSection" />
</configSections>

This must be the first entry in web.config, directly under the <configuration> element.

The <wsgi> element

All of the configuration options for NWSGI are under the <wsgi> element. The first ones we will look at are specified as attributes of the <wsgi> element: enableExtensions, adaptiveCompilation and frames.

enableExtensions

Enable extensions to NWSGI. Relying on these will make your application nonportable to other WSGI implementations. Required to use ASP.NET's built-in Session support, for example. Choices: true, false.

compilation

Control how IronPython compiles code. Choices: Adaptive, Compiled.

frames

Control sys._getframe. Enabling sys._getframe causes a performance hit. Choices: Off, On, Full.

Example:

<wsgi adaptiveCompilation="false" frames="Full" />

Adding Python Search Paths

If your application needs to access Python modules that are not in the default NWSGI search paths (~/, ~/Bin/ and ~/Bin/Lib/), you can use the <pythonPaths> element to add them. The <pythonPaths> element contains a set of <path> elements that are added (in order) to sys.path before the application is run. The <path> element has one required attribute – path – that contains the full path to a folder on disk containing the modules you want to use.

Example:

<wsgi>
    <pythonPaths>
        <path path="C:\django" />
    </pythonPaths>
</wsgi>

Adding WSGI Environment Variables

Most applications will require some configuration of their own; these options are passed in the environ parameter of the WSGI application. You can add entries to this list by using the <wsgiEnviron> element, which contains a set of <variable> elements. Each <variable> element has two required attributes, name and value.

Example:

<wsgi>
    <wsgiEnviron>
        <variable name="trac.env_path" value="C:\trac\projects" />
    </wsgiEnviron>
</wsgi>

Advanced Script Usage

There are cases where you may need to configure how NWSGI handles a particular script file. In these cases you need a virtual script mapping; NWSGI will always check script mappings before looking for a file on disk. Not surprisingly, these are stored in the <scriptMappings> element, which contains a set of <scriptMapping> elements. There are three parts to a script mapping:

scriptName – required

The name of the virtual script; overrides any scripts on disk.

physicalPath

The path to the actual file to execute. This can be a full path (i.e. C:\myapp\myapp.wsgi) or an app-relative path (i.e. ~/scripts/myapp.wsgi).

callable

If physicalPath is set, the name of the callable object in that file; if it is not set, an entry point for the application.

Entry points are used to specify a particular function in a Python module on sys.path (see the <pythonPaths> element above for how to control sys.path). For example, for Django the entry point would be "django.core.handlers.wsgi.WSGIHandler()". This mimics the standard Python syntax for creating an object; it is this object that is used as the application. If, instead, the application provides a function (like trac), the entry point would be "trac.web.main.dispatch_request".

Example:

<scriptMappings>
    <scriptMapping scriptName="myapp.wsgi" physicalPath="C:\myapp\myapp.wsgi" callable="myapp" />
</scriptMappings>

Wildcard Settings

When IIS is configured in wildcard mode, all requests for an application are passed to NWSGI, instead of only requests for .wsgi files. You can enable this by setting path="*" in the handler/httpHandler definition. Because NWSGI normally expects there to be a .wsgi file in the URL, but in wildcard mode there isn't one, the <wildcard> element is required to tell NWSGI what script to execute. The available settings – physicalPath and callable – have the same meaning as the same settings for script mappings (above). If the <wildcard> element is present, the <scriptMappings> element is ignored.

Example:

<wildcard physicalPath="C:\HelloWorld\hello.wsgi" callable="hello" />

Conclusion

NWSGI is extremely configurable; in many cases, you don't even need a .wsgi file. That said, the configuration can be complex, so please ask if you run into any problems. A later post will go into more detail on the mechanics of callables.

NWSGI 2.0: Configuration Changes

One of the two areas to see the most changes in NWSGI 2.0 was its configuration. Some of the options in v1.0 were overloaded in strange ways; even I wasn’t entirely sure what some combinations would produce. There was also the small problem of where to put the configuration section; it varied depending on how NWSGI was installed. For v2.0, both problems needed to be fixed. Doing so required a re-think of how configuration was handled.

Configuring Wildcards

In v1.0, configuring NWSGI to operate in wildcard mode required setting options that were designed for other things in particular ways that made no sense. In v2.0, there is a special <wildcard> element under <wsgi> (replacing the wildcardModule option) that is specifically used for configuring these settings. If the <wildcard> element exists, then NWSGI will ignore the <scriptMappings> element.

The callable option has also been removed from the <wsgi> element as it is no longer necessary (it defaults to “application”). To specify a custom callable name for a script, create a <scriptMapping>.

Where to put it?

The first problem was to put the <wsgi> section in the same place all the time. In v1.0 it could be directly under <configuration> (if you were using xcopy deployment) or under <system.webServer> (if you installed it on IIS7). The two could not necessarily coexist in the same web.config file, either! IIS 7 uses a different configuration assembly (Microsoft.Web.Administration) than ASP.NET (System.Configuration), and they’re no compatible, to boot.

The fix is relatively simple: rather than trying to use the appropriate configuration system from the handler, v2.0 always uses System.Configuration. This means that the <wsgi> section is always directly under <configuration>. The only downside is that the section always has to be declared at the top of web.config:

<configSections>
    <section name="wsgi" type="NWSGI.WsgiSection, NWSGI, Version=1.99.1.0, Culture=neutral, PublicKeyToken=41e64ddc1bf1fc86" />
</configSections>

New Options

There are also two new options: compilation and frames. The compilation option controls how IronPython generates code. The frames option is used to control sys._getframe; as tracking frames incurs a performance hit, it must be explicitly enabled.

Wrapping Up

The changes to configuration (which will break any existing configurations) are the main reason for bumping the version to 2.0. The clean up of the options resulted in much cleaner implementations for the management and especially for the handler itself. As always, if you have any comments, let me know.

NWSGI 2.0: Installation & Usage

There are two ways to use NWSGI: by installing it or by using xcopy deployment. Each method has its advantages and disadvantages, but their functionality is the same.

Installation

The quickest way to get NWSGI up and running is to use the MSI installer. This will add NWSGI to the GAC, making it available for all applications. If you are running IIS 7, it will also install the IIS 7 management components. The installer will also put a copy of the NWSGI assembly along with the README and HelloWorld example in C:\Program Files\NWSGI. This is the preferred approach, unless you do not have administrator rights to the server and cannot convince the server administrator to install it.

xcopy Deployment

The term “xcopy deployment” refers to the Windows xcopy command, which is used to (among other things) copy entire directory trees. Using this method is useful if you can’t (or don’t want to) use the installer package. If this is the case, download the .zip package instead of the MSI installer. To use this method, simply copy NWSGI.dll into your application’s Bin/ directory, and then copy the whole application folder to the server.

Setting Up the Hello World Example

Once you have decided which method you are going to use, the easiest way to check that it is working is to use the HelloWorld example that is included with both packages. I’m only going to describe the most common configuration, and I’m also going to assume you have access to the server; if you have a custom setup (such as a hosted site), just ask.

1. Copy the HelloWorld sample folder to C:\inetpub\wwwroot.
2. Mark C:\inetpub\wwwroot\HelloWorld as an application.
   • IIS 6/7: From the IIS management UI, mark the folder as an application.
   • IIS 7: From an elevated command prompt, run: appcmd add app /site.name:"Default Web Site" /path:/HelloWorld /physicalPath:C:\inetput\wwwroot\HelloWorld
3. Visit http://localhost/HelloWorld/hello.wsgi.

If the browser says “Hello, World!”, you’re good to go! If not, please ask in the forum, and I’ll try to help you out.

NWSGI 2.0 Beta 1

NWSGI 2.0 Beta 1 is now available. This is a major update to NWSGI that makes it easier to install and use and dramatically improves the management story.

So, what happened to 1.1?

Originally, NWSGI 1.1 was going to be a refresh of v1.0 that supported IronPython 2.6 and fixed a few bugs. NWSGI 2.0 was going to completely overhaul large parts of the management UI and make some deeper changes to the handler itself. However, I got a week off with some rain and v2.0 progressed much quicker than expected; v1.1 was obsolete before it was even released.

So, v1.1 was dropped and v2.0 will be the first IronPython 2.6-compatible release. It’s easy to make breaking changes when you have no customers :).

What’s New

• Easier configuration. In v1.0 there were different ways to configure NWSGI depending on your configuration. Now there is only one way to do it. Some of the more ambiguous options have also been cleaned up, and new ones added to support IronPython 2.6.
• Better management UI. IIS 7 includes a rich UI framework for developing management extensions, but doesn’t have much documentation on it. Some spelunking with reflector showed me better ways to do pretty much everything; not much of the v1.0 management code is left.
• Performance improvements. NWSGI will now cache the WSGI callable instead of reloading it each time.
• Other misc. fixes. I really need to get into the habit of adding a Codeplex issue for each bug so that I can keep track of what I’ve fixed.

Give it a spin

The Codeplex wiki needs to be updated to reflect the new documentation, but the included readme is up to date. I’ll also be posting a series of blogs posts over the next few weeks that will explain all the features of NWSGI and provide some helpful examples. Remember, this is a Beta release, so please let me know if you encounter any issues.