NWSGI 1.0

I'm very pleased to announce the release of NWSGI 1.0. NWSGI is an implementation of the WSGI specification (PEP 333) for IronPython. It is implemented as an ASP.NET HttpHandler and is supported on IIS 6 and IIS 7 (and should work on any other ASP.NET-capable web server). NWSGI allows the integration of Python applications with existing ASP.NET facilities such as caching, sessions, and user management.

This release does not mean that any Python application will work. The underlying IronPython 2.0 engine is not yet complete. It is a nearly-complete implementation of the NWSGI specification, and any app that takes IronPython's limitations into account should work just fine.

NWSGI fully supports xcopy deployment and medium trust environments (with the caveat that parts of the Python standard library are not supported). There is also an installer available that provides a managment UI for IIS 7.

NWSGI requires IIS (6 or 7), IronPython 2.0, and .NET Framework 2.0 SP1. All downloads include the NWSGI assembly, basic documentation, and a HelloWorld sample. The installers also include the IIS management UI.

Documentation is currently a weak spot, but I hope to improve that in the near future. In the meantime there is a README included, as well as some basic documentation at the NWSGI website.

easy_install on IronPython, Part Deux

Getting easy_install working for IronPython will be a big win for the IronPython ecosystem, and as I mentioned last time, the lack of zlib is really the only thing holding it back. Well, IronPython.Zlib solves that problem, so what else is holding up easy_install?

The big thing that's missing for setuptools is zipimport support, which is used to implement the Egg packages. To get things working, a stub zipimport module is needed with the following contents:

zipimporter = None

Next, make sure you are using at least Python 2.5.2 standard library (I'm not sure what version comes with IronPython 2.0, but it should be at least that new). The version of tarfile shipped with 2.5.0 doesn't seem to work at all.

Now some changes need to be made to the Python standard library. First up, since _winreg.error is not defined, distutils/msvccompiler.py must be patched to use WindowsError instead. We can't build C extensions anyway (or rather, there would be no point), but setuptools will still try. Then, py_compile.py needs to be changed so that the compile() funtion, well, doesn't – put "return" as the first statement, as IronPython can't create .pyc files (yet). Finally (and this is an IronPython.Zlib problem), do not throw if the CRC-32 check fails in gzip.py. A patch for all of these is available.

The next big changes is to implement binascii.crc32, which is used by the zipfile module. easy_install always tries to build an Egg, which requires zipfile (an Egg is just a .zip with some special files). This requires recompiling IronPython.Modules. I used the same (slightly questionable) CRC-32 implementation I used for IronPython.Zlib.

Once you get this far, easy_install can download, extract, and build a Python library (I used simplejson for my tests).  Installation is still a problem because pkg_resources is looking for an Egg, which require zipimport, which was blanked out in the first step (it's almost poetic, really). Getting zipimport working should be the last major hurdle.

Solving the Zlib Problem – IronPython.Zlib

IronPython 2.0 does not include an implementation of the zlib module, which handles data compression. In CPython the zlib module is a C module, which it can't be used from IronPython. It is also required by several standard Python modules, include tarfile, gzip, and zipfile, all of which handle compressed archives. Thus, for setuptools - particularly, easy_install and zipimport (Eggs) - to work, there must be a fully working zlib module.

FePy includes a partial implementation built on .NET's DeflateStream, but it is missing support for the decompressobj API that is used by gzip and zipfile. Supporting this API using DeflateStream appears to be impossible, as it expects a few implementation details of the zlib implementation.

That left two options for implementing zlib: P/Invoke, and ComponentAce's zlib.net. P/Invoke had a couple of issues: first, native-managed interop is hard; second, handling 32-bit and 64-bit systems seamlessly adds a lot of extra code (I use Vista x64, so I ran into this almost immediately). Zlib.net is an open source (BSD-style license), pure .NET implementation of zlib; this avoids both native-managed interop and 32/64-bit issues. It has an interface nearly identical to zlib, which means that it could be used to mimic the Python C library, which is exactly what I've done.

IronPython.Zlib is an implementation of the zlib module for IronPython using zlib.net. Both binary and source are available under the MS-PL license. To use it create a "DLLs" directory under your IronPython implementation and drop IronPython.Zlib.dll into it.

It passes most of the Python zlib tests and all of the gzip tests. There are a few issues with the CRC-32 implementation that the tests didn't catch but real-world usage did, but I haven't figured out how to fix them yet.

There are still some issues with setuptools (the zipfile module uses binascii.crc32, which is not yet implemented) that I'll detail in a later post, but it almost works.

Update June 27, 2009: Uploaded a new version that fixes crc32 bugs by forwarding to the now-implemented but 2.6-only binascii.crc32.

Django + IronPython

Update: See my newer post on how to run the Django test suite on IronPython 2.6.

I've had a couple of people ask about a Django + IronPython "Getting Started". This has been on my TODO list for a while; I just haven't gotten around to it. I'll try to do something over the next couple of weeks.

A couple of tips to anyone trying in the meantime:

• Get the SQLite provider from FePy SVN; it's the only one that I know works
• If manage.py has problems, file (or vote on) bugs, and use Python for manage.py instead
• Set the session backend to use the caching system, and set your cache backed to be in-memory (you can try to use the ASP.NET backends I posted earlier, but no guarantees)
• It probably won't work 100%, but I did work my way through the tutorial – except the admin system

NWSGI 0.7 Released

NWSGI 0.7 is now available on Codeplex. This release mainly deals with cleaning up some rough edges and incompatibilities with PEP 333, although there are some new features. If you used the installer for NWSGO 0.6, you must manually uninstall it before installing NWSGI 0.7.

Session Support

ASP.NET HttpHandlers must inherit from IRequiresSessionState¹ in order to have access to the Session state. This has been added to NwsgiHandler. Now, HttpContext.Session can be used to access the Session state. However, because the Session module adds a cookie, the entire response must be buffered, which is against the WSGI spec. See the section on Extensions for more details.

¹ Why did they have to add the first "s"? IRequireSessionState is too clever to pass up.

Proper Unicode Handling

IronPython does not distinguish between str and unicode (and neither does Jython). WSGI requires apps to return str objects, but allows Unicode if there is no other option and only the bottom 8 bits are used. Previously, NWSGI used the incorrect encoding (and not even consistently; I had both ASCII and UTF-8 in different places). PEP 333 specifies ISO 8859-1 (aka Latin1) for all Unicode conversions, as is maps precisely to the first 256 Unicode characters and thus preserves raw bytes (ASCII and UTF-8 mappings are required to move things around).

As a result, raw files can now be passed through NWSGI without the use of wsgi.file_wrapper, such is when generating images on the fly.

Wildcard Mappings

All HttpHandlers must have a target extension so that they can be used; NWSGI uses *.wsgi. This results in URLs such as http://www.example.com/hello/hello.wsgi/Jeff. There are two ways to handle this: URL rewriting, which is the preferred way, and wildcard mapping, which is more of a last resort as it can hurt performance for static files.²

By setting the handler path to "*", all file requests will be sent through NWSGI. To tell NWSGI that this is what you want, set the wildcardModule attribute like so:

<wsgi wildcardModule="~/hello.wsgi">

The equivalent URL would be http://www.example.com/hello/Jeff. NWSGI will handle all requests to that app by remapping SCRIPT_NAME to the application (/hello) and PATH_INFO to the remainder (/Jeff).

The wildcardModule attribute can be either absolute (C:\…) or app-relative (~/…). Using wildcardModule will bypass any script mappings, as they are unnecessary.

² One option is to move static files to a separate virtual directory or even separate server that does not use NWSGI.

WSGI Extensions

NWSGI implements a couple of non-conforming extensions to WSGI. In the interest of compatibility, these are off by default. To enable them, use the enableExtensions attribute:

<wsgi enableExtensions="true">

The current extensions are:

• nwsgi.context – provide access to the HttpContext via environ["nwsgi.context"]. This is somewhat moot as it can always be reached from HttpContext.Current.
• Responses are not flushed – PEP 333 requires the response to be flushed after every iteration of the WSGI callable. This breaks the use of ASP.NET Sessions, and possibly other IIS modules.

IronPython Version

NESGI 0.7 is linked against IronPython 2.0RC2, but RC1 should work as well – the assembly versions are the same. This hasn't been tested, however.