easy_install on IronPython

setuptools is a critical piece of the Python ecosystem: it extends distutils, but more importantly it provides easy_install, which is used by almost every Python library and application to handle installation. Making easy_install work in IronPython would make it far easier to download and install Python libraries.

The good news is that it's not too hard to get going; there are only a couple of bugs that need to be worked around. The bad news is that it can't work with any of the files that are downloaded, as the zlib module is shot.

This is all on IronPython 2.0b4.

Starting Off

First off, get the trunk version of setuptools: http://svn.python.org/projects/sandbox/trunk/setuptools. Under the checkout, run "python setup.py develop" (that will hopefully use ipy soon as well). Then, apply the following small patch:

==========================================================--- pkg_resources.py    (revision 65738)
+++ pkg_resources.py    (working copy)
@@ -2066,9 +2066,10 @@
             return str(self)

     def __str__(self):
-        try: version = getattr(self,'version',None)
-        except ValueError: version = None
-        version = version or "[unknown version]"
+        version = "[unknown version]"
         return "%s %s" % (self.project_name,version)

     def __getattr__(self,attr):

This is to work around #17810.

From your Python standard library, edit urllib2.py at line 1096 as follows:

+        fp = socket._fileobject(r, close=True)
-        fp = socket._fileobject(r)

The is work around #17894.

Under your IronPython\Lib directory, add a site-packages directory.

The zlib problem

This will allow you to run "ipy easy_install.py <package>". It will download the file, but fail (with a misleading error; there's an exception being swallowed somewhere) because there is no zlib module, and the one FePy is incomplete. Specifically, it is missing zlib.decompressobj.

I looked at implementing zlib.decompressobj, but it doesn't look like it will be easy with the .NET DeflateStream API.

Summary

easy_install can be made to work, but you can't do anything useful with it until the zlib module is fully implemented.

NWSGI 0.4 Released

To coincide with the release of IronPython 2.0b4, NWSGI(http://codeplex.com/NWSGI) has been updated to version 0.4. This version also brings improvements to configuration and performance.

Download NWSGI 0.4.

Configuration

This version brings major improvements to configuration. Previously, the only way to configure NWSGI was to use some undocumented AppSettings parameters. Now there is a full ConfigSection that changing NWSGI's behaviour a breeze. More information on the new configuration options is available here.

Performance

Older versions on NWSGI rebuilt the ScriptRuntime and ScriptEngine on each request; this is extremely slow. Now, each the ScriptRuntime and ScriptEngine and created once and each request is executed in its own ScriptScope. This saves some work on each request, but it also means that imported modules are cached instead of reloaded on each request.

This has two consequences: requests are processed much quicker, but the server must be manually recycled/restarted if the Python code changes. As IronPython's Achilles heel is still the slow import of Python modules, this is a big win for performance, but it does slow down development and debugging.

Conclusion

I'll be doing some Django testing (as well as some other apps) on 2.0b4, and I hope to be able to post a workable patch for Django soon.

Django on IronPython: On Databases

Django pretty well requires a database to do anything useful. It includes backends for a number of databases, such as mysql, portgres, and sqlite. Seeing as sqlite is by far the easiest to install, I started there. Using the dbapi module form FePy as a starting point, I extended the sqlite module to make it work for django (and made some tweaks to dbapi.py in the process).

There were a few hurdles to jump through, but for the most part it worked. I was able to run though the Django tutorial and build the complete 'poll' application. Except for some double-escaping on the admin page and a stupid bug in NWSGI, it works.

From there I started trying to get MS SQL Server working. This one is what most people would be using anywhere, and it doesn't require an extra assembly. For the most part it works, as long as you insure that all of the cursors are closed. If not, it doesn't like to work. The SQL Server classes really don't like having a DataReader open when committing a transaction. Unfortunately, manually closing cursors isn't going to scale to all of Django, so another approach will be necessary.

Using the sqlite3 driver requires System.Data.SQLite. Make sure you use the correct architecture; on Vista 64, IronPython requires the x64 driver.

Using the mssql driver requires code from the django-mssql project, with a minor change to base.py to load the mssql driver instead of the internal one.

Download sqlite3 and mssql drivers.