Tuesday, June 25, 2013

IronPython 3 TODO

This is my own list of things I want to see in IronPython 3 (after I get 2.7.4 out). It’s unlikely all of them will make 3.0 (which I’m targeting for PyCon 2014 next April), but hopefully most of them will.

Python 3 Features

Obviously, this is the most important thing. A couple are already implemented in branches (function annotations and part of keyword-only args), a few are relativelyeasy (metaclass changes, removal of old-style classes), and a few are hard (nonlocal, super(), yield from).

The first step will be to bring the new standard library and work from there. In addition, any changes needed to make the 3.x stdlib work on IronPython will be rolled into CPython so that we don’t have to maintain our own fork, with all of the work that that entails.

Better Test Coverage

IronPython’s test are currently a mess: they take too long to run and a huge number don’t even pass. Also, the test runner only works on Windows. It will need to be heavily modified (or probably replaced) with one that is more portable. It would also be nice to be able to generate coverage metrics, and be able to mark tests as “expected failure” so that TeamCity will actually build and run the tests successfully.

More Platforms

Windows desktop/server is no longer the only game in town. IronPython already works well on Mono, but without test coverage it’s hard to know how well. On top of that, the tablet/phone market is huge and getting bigger. Xamarin’s wonderful tools will make Android and iOS ports possible, and I’ve seen several requests for Windows Phone 8 and Windows Store (“Metro”) support as well, but they may not happen without someone volunteering to maintain them. Likewise, support for Silverlight will probably be dropped unless someone volunteers to maintain it.

The current code base is a mess of FEATURE_ and platform #ifdefs that is rather hard to maintain. There is the concept of a Platform Adaptation Layer (PAL), but it doesn’t get used everywhere. I’m going to see how feasible it is to expand the PAL to get rid of as many #ifdefs as possible.

At first each platform will only support embedding IronPython, but will eventually be extended to support building apps in pure Python.

“Static” Compilation and MSBuild support

IronPython currently has pyc.py to generate executables and DLLs, but it’s a bit clunky, only supports .NET 4, and is missing some useful features like arbitrary resources. Improvements to pyc.py will add those missing features to generate truly standalone executables.

The other issue is that the DLLs it generates are not usable from any .NET language; they must be loaded in an IronPython host. “Static” compilation will allow DLLs to be generated with actual .NET classes that can be reflected over and thus will be useful as plugins (for e.g. MSBuild or IIS) or to build pure-Python Andoid/iOS/WP8/Metro apps, all of which are basically plugins as well. Python classes just need some special adornment and pyc.py will pick them up and compile them to real types (or something similar):

import clr
of = clr.of

class RealClass(metaclass=clr.object):
    def Foo(self, x : of(int), y : of(str)) -> float:

This is both a lot and a little bit of work; the code to do the type generation is already there, but it makes some assumptions about global state that need to be pulled apart and refactored.


OK, so there’s a lot there for nine months. And, keep in mind, this is just what I want to work on. However, I think this list will make sure that IronPython stays viable in the future. If you have any other suggestions, let me know.

As always, if you’re interested in helping, get in touch. I’m more then willing to help anyone get started. The codebase is intimidating at first, but once you got over the initial learning wall it’s not so bad.