[Numpy-discussion] Openmp support (was numpy's future (1.1 and beyond): which direction(s) ?)
Mon Mar 24 12:12:51 CDT 2008
Matthieu Brucher wrote:
> It is a real problem in some communities like astronomers and images
> processing people but the lack of documentation is the first one,
> is true.
> Even in those communities, I think that a lot could be done at a
> higher level, as what IPython1 does (tasks parallelism).
Well it is not that easy. We have several numpy code following like this :
1) open an large data file to get a numpy array
2) perform computations on this array (I'm only talking of the numpy
part here. scipy is something else)
3) Write the result is another large file
It is so simple to write using numpy :)
Now, if I want to have several exe, step 3 is often a problem. The only
simple way to speed this up is to slit step 2 into threads (assuming
that there is no other possible optimisation like sse which is false but
out of the scope of numpy users).
Using C, we can do that using OpenMP pragma. It may not be optimal but
it radio speedup/time_to_code is very large :) Now, we are switching
from C to numpy because we cannot spend that much time to play with
gdb/pointers to open an image anymore.
ps : I have seen your blog and you can send me an email off line about
this topic and what you are doing :)
More information about the Numpy-discussion