[SciPy-User] [mpi4py] MPI, threading, and the GIL
Fri Sep 9 14:45:32 CDT 2011
*More specifically, our iterative algorithm needs to send data from
rank N to rank N+1, but the rank N+1 processor doesn't need this data
immediately - it has to do a few other things before it needs it. For
each MPI process, I have three threads: one thread for computations,
one thread for doing MPI sends, and one thread for doing MPI receives.
This is not idiomatic MPI. You can do the same thing with a single thread
(and avoid GIL issues) by posting non-blocking sends and receives
(MPI_Isend/MPI_Irecv) when you have the data to send and then issuing a
'wait' when you need the data to proceed on the receiving end.
On Fri, Sep 9, 2011 at 10:41 PM, Matthew Emmett <email@example.com> wrote:
> Hi everyone,
> I am having trouble with MPI send/recv calls, and am wondering if I
> have come up against the Python GIL. I am using mpi4py with MVAPICH2
> and the threading Python module.
> More specifically, our iterative algorithm needs to send data from
> rank N to rank N+1, but the rank N+1 processor doesn't need this data
> immediately - it has to do a few other things before it needs it. For
> each MPI process, I have three threads: one thread for computations,
> one thread for doing MPI sends, and one thread for doing MPI receives.
> I have set this up in a similar manner to the sendrev.py example here:
> The behavior that I have come across is the following: the time taken
> for each iteration of the computational part varies quite a bit. It
> should remain roughly constant, which I have confirmed in other tests.
> After all, the amount of work done in the computational part remains
> the same during each iteration. It seems like the threads are not
> running as smoothly as I expect, and I wonder if this is due to the
> GIL and my use of threads.
> Has anyone else dealt with a similar problem?
> I have a slightly outdated F90 implementation of the algorithm that
> isn't too far behind its Python cousin. I will try to bring it up to
> date and try the new communication pattern, but it would be nice to
> stay in Python land if possible.
> Any suggestions would be appreciated. Thanks,
> You received this message because you are subscribed to the Google Groups
> "mpi4py" group.
> To post to this group, send email to firstname.lastname@example.org.
> To unsubscribe from this group, send email to
> For more options, visit this group at
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the SciPy-User