[Numpy-discussion] Proposed Roadmap Overview

Dag Sverre Seljebotn d.s.seljebotn@astro.uio...
Mon Feb 20 11:18:59 CST 2012


On 02/20/2012 08:55 AM, Sturla Molden wrote:
> Den 20.02.2012 17:42, skrev Sturla Molden:
>> There are still other options than C or C++ that are worth considering.
>> One would be to write NumPy in Python. E.g. we could use LLVM as a
>> JIT-compiler and produce the performance critical code we need on the fly.
>>
>>
>
> LLVM and its C/C++ frontend Clang are BSD licenced. It compiles faster
> than GCC and often produces better machine code. They can therefore be
> used inside an array library. It would give a faster NumPy, and we could
> keep most of it in Python.

I think it is moot to focus on improving NumPy performance as long as in 
practice all NumPy operations are memory bound due to the need to take a 
trip through system memory for almost any operation. C/C++ is simply 
"good enough". JIT is when you're chasing a 2x improvement or so, but 
today NumPy can be 10-20x slower than a Cython loop.

You need at least a slightly different Python API to get anywhere, so 
numexpr/Theano is the right place to work on an implementation of this 
idea. Of course it would be nice if numexpr/Theano offered something as 
convenient as

with lazy:
     arr = A + B + C # with all of these NumPy arrays
# compute upon exiting...

Dag


More information about the NumPy-Discussion mailing list