[Numpy-discussion] Numpy and OpenMP
Sat Mar 15 15:51:40 CDT 2008
Robert Kern wrote:
> On Sat, Mar 15, 2008 at 2:48 PM, Gnata Xavier <firstname.lastname@example.org> wrote:
>> Numpy is great : I can see several IDL/matlab projects switching to numpy :)
>> However, it would be soooo nice to be able to put some OpenMP into the
>> numpy code.
>> It would be nice to be able to be able to use several CPU using the
>> numpy syntax ie A=sqrt(B).
>> Ok, we can use some inline C/C++ code but it is not so easy.
>> Ok, we can split the data over several python executables (one per CPU)
>> but A=sqrt(B) is so simple...
>> numpy + recent gcc with OpenMP --> :) ?
>> Any comments ?
> Eric Jones tried to use multithreading to split the computation of
> ufuncs across CPUs. Ultimately, the overhead of locking and unlocking
> made it prohibitive for medium-sized arrays and only somewhat
> disappointing improvements in performance for quite large arrays. I'm
> not familiar enough with OpenMP to determine if this result would be
> applicable to it. If you would like to try, we can certainly give you
> pointers as to where to start.
Well of course if the arrays are too small it will be slower but it *is*
much faster on large arrays.
In many cases, there is no need lock/unlock : Look at A=sqrt(A) : it is
obvious to speed-up such a compuation in pure C using OpenMP. From a
simple minded point of view, I would say that somewhere in numpy, there
should be such a C loop.
Why do we really (IMHO) need that ? Because "all" the machines (even
laptops) are now multicore/cpu.
Using IDL, it is possible to develop a quite large image processing
project working on large images on a 8core machine without *any*
konwledge of semaphore/lock.
All this piece of software uses the simple syntaxe of IDL (ugly ones
compare to numpy :))
I used to be very sceptical about the performances but I had a look and
*it just works well*. It scales just nicely up to 6 cores.
Small arrays computation shall *not* be threaded, large ones should be
if we look at the multicores trend.
Any comments ?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Numpy-discussion