[SciPy-User] a rhetorical question on multithreading..

Robert Kern robert.kern@gmail....
Wed Oct 14 14:40:54 CDT 2009


On Wed, Oct 14, 2009 at 12:47, Gustaf Nilsson <gustaf@laserpanda.com> wrote:
> Hi
> I guess if it was a good idea, then it wouldve been done already but...
> As the numpy/scipy underlying parts are written in C (?), wouldnt it make
> more sense to make them multithreaded by themselves, instead of having us
> hack around it (and GIL) in python?
> Or have i misunderstood?

Making something multithreaded isn't really a specific thing. There
are many strategies one can apply and some of them involve
multithreading. Unfortunately, many of those strategies also need
tuning for the particular platform, which is something that is
difficult to do for a general library like numpy. Additionally, some
of the most promising parallelism strategies for your code operate at
a higher level than the C-coded operations in numpy; just making the
ufuncs and C-coded routines multithreaded gives you a smaller win
making the effort less worthwhile. Furthermore, at the C level,
threading introduces platform-specific code that make things more
difficult.

Which isn't to say that we haven't tried. Take a look at the multicore/ branch:

  http://svn.scipy.org/svn/numpy/branches/multicore/

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


More information about the SciPy-User mailing list