[IPython-user] pushing/pulling objects between engines
Tue Oct 2 15:41:02 CDT 2007
On 10/2/07, Jonathan Taylor <email@example.com> wrote:
> Is it possible to send information between two engines using the
> I can imagine wanting to do something like the following between two
> engines i and j
> a_i, a_j = a_j, a_i
> (not proper python syntax of course).
> I didn't see an example of this in the tutorial.... other than
> >>> a_i, a_j = rc.pull(i, 'a'), rc.pull(j, 'a')
> >>> rc.push(j, 'a'=a_i); rc.push(i, 'a'=a_j)
> Any suggestions?
At the moment unfortunately that's it, but with a caveat (see below).
Currently the engines all connect to the controller explicitly, and
they don't listen on any other ports than that connection, so they
have no mechanism for sending directly data to one another.
The caveat is that the system is fully compatible with MPI, so you can
start your group of engines as part of a single MPI world, and at
any point tell your engines to communicate with one another via MPI.
We need to look into cleaner ways to integrate inter-engine
communications in the future, but for now the MPI solution might do
the trick for you. It also has the advantage of being very efficient,
at the cost of a less convenient API>
More information about the IPython-user