[SciPy-user] Python on Intel Xeon Dual Core Machine
Wed Feb 6 12:58:15 CST 2008
Has anyone played with openmp using ctypes or weave?
On Feb 6, 2008 1:01 PM, Young, Karl <email@example.com> wrote:
> > Parallel programming is a massive, complicated field, and many
> > high-powered software tools exist to take advantage of it.
> > Unfortunately, python has a limitation in this area:
> the Global
> > Interpreter Lock. Basically it means no two CPUs can be running python
> > code at the same time. This means that you get no speedup at all by
> > parallelizing your python code - with a few important exceptions:
> > while one thread is doing an array operation, other threads can run
> > python code, and while one thread is waiting for I/O (reading from
> > disk, for example), other threads can run python code. Parallel python
> > is a toolkit that can avoid this problem by running multiple python
> > interpreters (though I have little experience with it).
> Well yes, but on a cluster (distributed memory) you can still take
> advantage of parallelization using python tools particularly
> if the parallelization is close to trivial (admittedly it doesn't sound
> like that is Lorenzo's situation). But I agree with everything
> Anne says in that parallel programming is a massively complicated area and
> it's really important to do things like profiling your code first.
> But I was (and am) fairly ignorant of many of the important details, other
> than realizing that unfavorable communication to computation
> ratios could kill you, and was still able to get close to linear speedup
> (though my problem, while not completely, was close to
> trivially parallelizable, i.e I just needed to pass a few things around
> between steps requiring independent chunks requiring long
> calculations). So I still think it's useful to do a quick and dirty
> estimate of communication/computation and if that looks favorable
> explore some "simple" parallel programming tools like pypar.
> SciPy-user mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the SciPy-user