by Kay Hayen for EuroPython 2012
With Nuitka, for the first time, there is a consequently executed approach to statically translate the full language extent of Python, with all its special cases, without introducing a new or reduced version of Python.
It is compiled, but with practically 100% compatibility. Function dictionaries work, code objects exist, frame stack works, exception tracebacks, eval, exec, closures, nested functions, meta classes, etc. it’s all there, and behaves identical.
First, I would like to start out and explain how I came to write a Python compiler, why I want it to be 100% compatible, and why I find deviations from Python unacceptable and out of scope.
Then I would like to describe where difficulties were in the implementation, what Python constructs surprised me, and where the mapping from Python to C++ left things to desire.
In this project, I learned a lot about Python, it wasn’t easy to get the full CPython test suite to run. In doing that, I have learned anecdotes and fine details of Python, that are normally hidden in daily programming, but are still useful to know.
Esp. the work on re-formulating “with” statements, “assert”, “while"/"for” as generic loops, etc. gives an interesting view on Python itself. And I would like to present it, also for the insight it gives on Python.
I will give an overview over newly developed infrastructure, aiming for type inference at compile time, and show existing stuff. I will try and explain, why I hope to have picked the right approach in this domain.
An interesting side game, is the approach to use XML representations of the internal node tree of Nuitka to discover regressions/changes in the optimizer.
Then I will also present a project road map, with the milestones for Nuitka, and why I believe this is the right plan, and how Nuitka is different from projects like “Cython” and “PyPy”.
To round it up, I would like to make a demonstration of Nuitka, and give an example for how easy it should be to contribute.
As this will be the first time, Nuitka is introduced the PyCON EU (it was only shown on PyCON DE 2011 so far). And to celebrate that, the current GPLv3 license will be lifted, and replaced with Apache 2.0 license (ASF), which is entirely liberal.
Actually, even now, I am still doing structural changes, e.g. the re-formulation work, which will make optimizations easier, and then I discover that I need more structural changes.
Answering the whole stdlib question is a bit tough. But "hg.exe" on Linux, is 24MB, so sizes are acceptable.
As for interesting blog subjects, well yes, but you have to be aware, that this is a spare time effort.
If you are interesting in my plans, you can find them in the Developer Manual, at http://www.nuitka.net/doc/developer-manual.html
There I detail how I intend to approach type inference.
Talking about that though, would have seemed to pre-mature. I believe my plan is half-sound and it appears to work out, but I don't want to talk about unfinished things.
In my first English talk, it seemed more appropiate to get out some kind of manifesto and explain the basic choices made by the project.
I hope, when the project advances, you will have a chance to see more beef to it.
I'm impressed with the amount of progress made so far by. Keep up the good work.
(I will agree that the PyPy benchmarks would be a good addition though. I look forward to seeing that)
The http://speedcenter.nuitka.net was setup quite short before the conference, and surely, it will see more benchmarks added as I find the time.
As for benchmarks, the PyPy ones, I will probably do them too. But I consider the comparison to ShedSkin with its demos to be far more interesting for my development.
Why? Because I will look at these programs as something where type inference absolutely can be done. Shedskin won't compile, if it's not possible.
And there I will be able to compare Shedskin and Nuitka and learn something from that.
Because benchmarks of all sorts, clearly are a must, once it's to be declared ready for the users. Until then, benchmarks should rather aid development.
One thing Guido wasn't at all taking into account, is that I have day job and family, and due to that, I can't just do everything that would be nice to have. But I have to make choices. And typically, I would rather just improve Nuitka.
But now that my blog is based on ReST, I expect to be able to post stuff from the developer manual, or generally make things public, once I write it up.
That developer manual, I made it only (and not only PDF as it just ed be just before the conference). It became possible thanks to Nikola.
So I realize, that while I have lots of interesting things to say, there is the issue, that actually doing it, was happening in ways, where nobody could see it. And I have worked to address that in my changing to Nikola, which will make it much more easy.
So well yeah. Don't forget, this was the first time you heard of Nuitka. Not the last time.
Check out that "ctypes" plan, in developer manual. It is my current state of toughts, for how to design the type inference.