After months of silence, the Pyston project -- a Dropbox-sponsored initiative to create a high-performance, JIT-compiling Python runtime -- has announced a new version.
The bad news: Pyston 0.5 is slower than previous versions. The good news: It's also more compatible.
According to a post on the official Pyston project blog, version 0.5 features support for reference counting ("refcounting"), the memory management methodology used by the Python's original CPython implementation. Originally, Pyston ran a more sophisticated tracing garbage collector, but it created serious compatibility problems with existing programs.
Rather than try to make Pyston's garbage collection system backward-compatible with the expectations of existing Python applications, Pyston's developers decided to ditch the tracing garbage collector altogether and go back to a refcounting system like the one Python originally employed.
One of Python's most important packages, NumPy, widely used in mathematical, scientific, and financial applications, can now run as-is in Pyston. Other libraries that Pyston's developers described as "tricky," such as cffi (the C Foreign Function Interface, a third-party system for linking Python and C libraries), also now work.
Steps forward and back
On the other hand, Pyston took a performance hit as a result of this change of horses. Pyston 0.5 is "about 10 percent slower" than version 0.4, but Pyston's developers are confident the problem isn't insurmountable. They say it's mainly a product of focusing first on getting refcounting running rather than optimizing it completely.
Some of the slowdown with NumPy, for instance, is due to calls from C code back into the Python runtime, which is not yet optimized in Pyston.
Performance will be a priority for version 0.6, say Pyston's developers, with an interim 0.51 release coming soon to apply quick-and-dirty performance fixes. Pyston currently supports only Python 2, but Python 3 compatibility is cited as a long-term goal.
More ways to go faster
Aside from Pyston, Python developers are exploring several strategies for a faster runtime. One, long established and widely adopted, is the PyPy runtime, which also uses JIT compilation. Another possibility is a series of projects that will attempt to speed up the stock CPython runtime -- not orders of magnitude faster, but incrementally so.
A third project, Pyjion, is a Microsoft initiative that adds a JIT interface to CPython. This way, accelerators can be plugged into the existing Python runtime, granting both high speed and strong backward compatibility. Microsoft has plans for a JIT based on the one in the CoreCLR that could be plugged into such an API.
Backward compatibility has been a lingering bugbear for speeding up Python. Many projects allow Python acceleration, typically at the cost of breaking existing Python applications and libraries. Or they involve changes to Python's syntax that yield the same net result. The Holy Grail of all the advantages and none of the drawbacks remains elusive -- though based on the enthusiasm shown by Pyston's developers and their cohorts, it's not out of reach.