[Numpy-discussion] Proposed Roadmap Overview

Travis Oliphant travis@continuum...
Mon Feb 20 22:04:00 CST 2012


Interesting you bring this up.   I actually have a working prototype of using Python to emit LLVM.   I will be showing it at the HPC tutorial that I am giving at PyCon.    I will be making this available after PyCon to a wider audience as open source.  

It uses llvm-py (modified to work with LLVM 3.0) and code I wrote to do the translation from Python byte-code to LLVM.   This LLVM can then be "JIT"ed.   I have several applications that I would like to use this for.   It would be possible to write "more of NumPy" using this approach.     Initially, it makes it *very* easy to create a machine-code ufunc from Python code.   There are other use-cases of having loops written in Python and plugged in to a calculation, filtering, or indexing framework that this system will be useful for.  

There is still a need for a core data-type object, a core array object, and a core calculation object.   Maybe some-day these cores can be shrunk to a smaller subset and more of something along the lines of LLVM generation from Python can be used.   But, there is a lot of work to do before that is possible.    But, a lot of the currently pre-compiled loops can be done on the fly instead using this approach.    There are several things I'm working on in that direction. 

This is not PyPy.   It certainly uses the same ideas that they are using, but instead it fits into the CPython run-time and doesn't require changing the whole ecosystem.     If you are interested in this work let me know.  I think I'm going to call the project numpy-llvm, or fast-py, or something like that.   It is available on github and will be open source (but it's still under active development). 

Here is an example of the code to create a ufunc using the system (this is like vectorize, but it creates machine code and by-passes the interpreter and so is 100x faster).  

from math import sin, pi

def sinc(x):
    if x==0:
        return 1.0
    else:
        return sin(x*pi)/(pi*x)

from translate import Translate
t = Translate(sinc)
t.translate()
print t.mod

res = t.make_ufunc('sinc')


-Travis



On Feb 20, 2012, at 10:55 AM, Sturla Molden wrote:

> Den 20.02.2012 17:42, skrev Sturla Molden:
>> There are still other options than C or C++ that are worth considering.
>> One would be to write NumPy in Python. E.g. we could use LLVM as a
>> JIT-compiler and produce the performance critical code we need on the fly.
>> 
>> 
> 
> LLVM and its C/C++ frontend Clang are BSD licenced. It compiles faster 
> than GCC and often produces better machine code. They can therefore be 
> used inside an array library. It would give a faster NumPy, and we could 
> keep most of it in Python.
> 
> Sturla
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20120220/15e08500/attachment-0001.html 


More information about the NumPy-Discussion mailing list