[SciPy-User] MPI, threading, and the GIL

Matthew Emmett memmett@unc....
Fri Sep 9 14:41:46 CDT 2011


Hi everyone,

I am having trouble with MPI send/recv calls, and am wondering if I
have come up against the Python GIL.  I am using mpi4py with MVAPICH2
and the threading Python module.

More specifically, our iterative algorithm needs to send data from
rank N to rank N+1, but the rank N+1 processor doesn't need this data
immediately - it has to do a few other things before it needs it.  For
each MPI process, I have three threads: one thread for computations,
one thread for doing MPI sends, and one thread for doing MPI receives.

I have set this up in a similar manner to the sendrev.py example here:

  http://code.google.com/p/mpi4py/source/browse/trunk/demo/threads/sendrecv.py

The behavior that I have come across is the following: the time taken
for each iteration of the computational part varies quite a bit.  It
should remain roughly constant, which I have confirmed in other tests.
 After all, the amount of work done in the computational part remains
the same during each iteration.  It seems like the threads are not
running as smoothly as I expect, and I wonder if this is due to the
GIL and my use of threads.

Has anyone else dealt with a similar problem?

I have a slightly outdated F90 implementation of the algorithm that
isn't too far behind its Python cousin.  I will try to bring it up to
date and try the new communication pattern, but it would be nice to
stay in Python land if possible.

Any suggestions would be appreciated.  Thanks,
Matthew


More information about the SciPy-User mailing list