NWSGI 1.1 Preview 1

A preview of NWSGI 1.1 is now available on Codeplex. This preview is linked against IronPython 2.6a1 so that I can start testing against 2.6 with enough lead time to actually get any bugs fixed before release :). There are also some enhancements to the configuration that can simplify some common scenarios; see the release notes or changes.txt for more information. I'll write up more detailed instructions on how to take advantage of those changes in the future.

IronPython in Action

IronPython in Action is an upcoming book from Manning Publications by Michael Foord and Christian Muirhead. It describes the intersection of the .Net Framework and IronPython with a large number of examples, covering a reasonably large swath of the .Net Framework: WinForms, WPF, ASP.NET, Silverlight, XML processing, and more.

IronPython in Action is exactly what it says on the tin: this book is a large collection of real-world(ish) examples showing how the .Net Framework can be used in conjunction with the Python language. To go with this book, I'd recommend a Python language reference and a reference for whichever .Net technologies you are working with. This book doesn't try to by the definitive reference for either Python or .Net; it focuses on the intersection of the two. For the most part there is very little mismatch between the two; C# (or VB) can be translated into Python and everything will (mostly) work fine.

The first part of the book gives a very quick overview of the history of Python, .Net, and IronPython; this is followed by a brief introduction to the Python language and how to access various .Net features (classes, enums, events, etc.) from IronPython.

The second part of the book is the creation of a simple WinForms application that is used to demonstrate some .Net framework classes and some Python design techniques and idioms. In particular, it covers the basics of WinForms programming and XML processing, along with some insights into patterns and unit testing in Python. The final chapter of this section (Ch. 8) covers Python metaprogramming and protocols, and some gory details about interacting with the CLR – in particular, arrays and reference parameters are tricky to use from the Python side.

Part 3 of the book covers a wide range of topics: WPF, systems administration, ASP.NET, database access, web services, and Silverlight. Again, none of these subjects are covered in detail – each could (and has!) fill a couple of volumes. Instead, the authors focus on getting everything going and covering some of the issues that may come up. The ASP.NET demonstration is particularly clever as it is simply a recasting of the example editor from Part 2 as a web site; the WinForms code had to be swapped for WebForms code, but none of the rest of the code (the controllers and the model) had to be changed. The database section uses PostgreSQL as the  database target; the only major difference to SQL Server would be the connection string (and some edge cases in the SQL).

The final section of the book covers embedding and extending IronPython. It is a nice, simple introduction to building a C# (or VB) IronPython extension, followed by demonstrating how to embed the IronPython engine into a .Net program. The embedding section, in particular, has some tricks that are hard to learn otherwise – the functionality is a bit spread out, and the current documentation is only passable. Thankfully, Ch. 15 will be unnecessary (or at least very different) with C# 4.0.

I wasn't really sure what to expect when given this book. Having read it, I now see that it is exactly what it says it is – IronPython, in action. For any serious work with IronPython (more than just using it as an alternative Python interpreter), than this book will save you a ton of learning time. The examples are clear, concise, and – most importantly – realistic. Even if you already know Python and .Net inside and out, but are new to IronPython, this book will save you some headaches learning about how they interact.

NWSGI 1.0

I'm very pleased to announce the release of NWSGI 1.0. NWSGI is an implementation of the WSGI specification (PEP 333) for IronPython. It is implemented as an ASP.NET HttpHandler and is supported on IIS 6 and IIS 7 (and should work on any other ASP.NET-capable web server). NWSGI allows the integration of Python applications with existing ASP.NET facilities such as caching, sessions, and user management.

This release does not mean that any Python application will work. The underlying IronPython 2.0 engine is not yet complete. It is a nearly-complete implementation of the NWSGI specification, and any app that takes IronPython's limitations into account should work just fine.

NWSGI fully supports xcopy deployment and medium trust environments (with the caveat that parts of the Python standard library are not supported). There is also an installer available that provides a managment UI for IIS 7.

NWSGI requires IIS (6 or 7), IronPython 2.0, and .NET Framework 2.0 SP1. All downloads include the NWSGI assembly, basic documentation, and a HelloWorld sample. The installers also include the IIS management UI.

Documentation is currently a weak spot, but I hope to improve that in the near future. In the meantime there is a README included, as well as some basic documentation at the NWSGI website.

easy_install on IronPython, Part Deux

Getting easy_install working for IronPython will be a big win for the IronPython ecosystem, and as I mentioned last time, the lack of zlib is really the only thing holding it back. Well, IronPython.Zlib solves that problem, so what else is holding up easy_install?

The big thing that's missing for setuptools is zipimport support, which is used to implement the Egg packages. To get things working, a stub zipimport module is needed with the following contents:

zipimporter = None

Next, make sure you are using at least Python 2.5.2 standard library (I'm not sure what version comes with IronPython 2.0, but it should be at least that new). The version of tarfile shipped with 2.5.0 doesn't seem to work at all.

Now some changes need to be made to the Python standard library. First up, since _winreg.error is not defined, distutils/msvccompiler.py must be patched to use WindowsError instead. We can't build C extensions anyway (or rather, there would be no point), but setuptools will still try. Then, py_compile.py needs to be changed so that the compile() funtion, well, doesn't – put "return" as the first statement, as IronPython can't create .pyc files (yet). Finally (and this is an IronPython.Zlib problem), do not throw if the CRC-32 check fails in gzip.py. A patch for all of these is available.

The next big changes is to implement binascii.crc32, which is used by the zipfile module. easy_install always tries to build an Egg, which requires zipfile (an Egg is just a .zip with some special files). This requires recompiling IronPython.Modules. I used the same (slightly questionable) CRC-32 implementation I used for IronPython.Zlib.

Once you get this far, easy_install can download, extract, and build a Python library (I used simplejson for my tests).  Installation is still a problem because pkg_resources is looking for an Egg, which require zipimport, which was blanked out in the first step (it's almost poetic, really). Getting zipimport working should be the last major hurdle.

Solving the Zlib Problem – IronPython.Zlib

IronPython 2.0 does not include an implementation of the zlib module, which handles data compression. In CPython the zlib module is a C module, which it can't be used from IronPython. It is also required by several standard Python modules, include tarfile, gzip, and zipfile, all of which handle compressed archives. Thus, for setuptools - particularly, easy_install and zipimport (Eggs) - to work, there must be a fully working zlib module.

FePy includes a partial implementation built on .NET's DeflateStream, but it is missing support for the decompressobj API that is used by gzip and zipfile. Supporting this API using DeflateStream appears to be impossible, as it expects a few implementation details of the zlib implementation.

That left two options for implementing zlib: P/Invoke, and ComponentAce's zlib.net. P/Invoke had a couple of issues: first, native-managed interop is hard; second, handling 32-bit and 64-bit systems seamlessly adds a lot of extra code (I use Vista x64, so I ran into this almost immediately). Zlib.net is an open source (BSD-style license), pure .NET implementation of zlib; this avoids both native-managed interop and 32/64-bit issues. It has an interface nearly identical to zlib, which means that it could be used to mimic the Python C library, which is exactly what I've done.

IronPython.Zlib is an implementation of the zlib module for IronPython using zlib.net. Both binary and source are available under the MS-PL license. To use it create a "DLLs" directory under your IronPython implementation and drop IronPython.Zlib.dll into it.

It passes most of the Python zlib tests and all of the gzip tests. There are a few issues with the CRC-32 implementation that the tests didn't catch but real-world usage did, but I haven't figured out how to fix them yet.

There are still some issues with setuptools (the zipfile module uses binascii.crc32, which is not yet implemented) that I'll detail in a later post, but it almost works.

Update June 27, 2009: Uploaded a new version that fixes crc32 bugs by forwarding to the now-implemented but 2.6-only binascii.crc32.