[SciPy-User] Multiprocessing and shared memory
Sun Oct 18 22:08:47 CDT 2009
Felix Schlesinger skrev:
>> with nogil:
> Well this only works if the parallel operation does not touch any
> python objects, right? Otherwise it would reenter the interpreter
> without the GIL and not be thread-safe anymore.
Yes and no. Cython will tell you if the GIL is required (you'll get an
You can use numpy ndarrays in nogil blocks.
import numpy as np
cimport numpy as np
cdef int n
cdef double rvalue = 0.1
cdef np.ndarray[double, ndim=1, mode='c'] array
array = np.zeros(10)
for n from 0 <= n < 10:
array[n] = rvalue
> In other words the
> whole algorithm would have to be moved into cython.
Only the worst bottlenecks.
> Is this approach
> stable in your experience?
Cython is a stabile approach. It is used by SciPy and Sage, and many
other projects. It will probably be used to port NumPy to Python 3. It's
not a toy.
You can prototype your code in Python. When you are done, you can add in
some type declarations here and there, compile, and get the same
performance as C or C++.
> It sounds like it would be easy to create
> race-conditions to me.
In my experience it is easier to get race conditions when using threads
in C, C++ or Java. Anything outside nogil blocks is serialized with
Cython. Since Cython is compiled, there are no thread switch in
Cython-land util you release the GIL manually. Anything not declared
nogil becomes thread-safe by default.
Also: You can run closures as threads in Python. Just put the loops you
want parallelized in a closure, and you have the same more or less the
same constructs as used by OpenMP or Apple's GCD.
More information about the SciPy-User