[SciPy-user] Multithreading cookbook entry
Thu Feb 21 08:58:32 CST 2008
I have no answer, but I want to add another question
to Ms. Archibald or anyone. Will the GIL prevent in
any way the threading of C extensions? That is, I
want to call a C extension in several threads and the
program will stay there doing a long calculation and
then return to Python when finished. Perhaps this is
obvious, but I admit I don't fully understand the GIL.
Thanks for any info.
I want to evaluate a function using a C extension
(implemented with ctypes) for several parameters in
the function. The parameters are in a list. Then I
use the handythread.py approach and for each thread
call the C extension function with a new parameter
value from the list and, when the thread returns, I
add the result (say, a float number) to a result list.
Will, the GIL let the threads run independently? I
hope my example is clear.
Thanks for any info.
--- Anand Patil <email@example.com>
> Hi all,
> I have a question primarily for Anne Archibald, the
> author of the
> cookbook entry on multithreading,
> I tried replacing the 'if name=='__main__' clause in
> the attachment
> handythread.py with
> from numpy import ones, exp
> def f(x):
> print x
> y = ones(10000000)
> and the wall-clock time with foreach was 4.72s vs
> 6.68s for a simple for-loop.
> First of all, that's amazing! I've been internally
> railing against the
> GIL for months. But it looks like only a portion of
> f is being done
> concurrently. In fact if I comment out the 'exp(y)',
> I don't see any
> speedup at all.
> It makes sense that you can't malloc simultaneously
> from different
> threads... but if I replace 'ones' with 'empty', the
> time drops
> precipitously, indicating that most of the time
> taken by 'ones' is
> spent actually filling the array with ones. It seems
> like you should
> be able to do that concurrently.
> So my question is, what kinds of numpy functions
> tend to release the
> GIL? Is there a system to it, so that one can figure
> out ahead of time
> where a speedup is likely, or do you have to try and
> see? Do
> third-party f2py functions with the 'threadsafe'
> option release the
> SciPy-user mailing list
-- Lou Pecora, my views are my own.
Looking for last minute shopping deals?
Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping
More information about the SciPy-user