[IPython-User] Parallel question: Sending data directly between engines
Fri Jan 6 22:05:24 CST 2012
Looks like there is another example in that same directory that creates
some convenience functions around inter-engine communication. I suspect it
will solve my problem.
On Fri, Jan 6, 2012 at 5:30 PM, Fernando Perez <email@example.com> wrote:
> Hi Matthew,
> On Fri, Jan 6, 2012 at 1:59 PM, Matthew Rocklin <firstname.lastname@example.org>
> > Hello,
> > What is the easiest way for me to send data directly between two ipython
> > engines?
> > I.e. if I'm running a "master" script on a particular machine and type in
> > something like the following
> > rc = p.Client()
> > e0 = rc
> > e1 = rc
> > e0['data'] = e1['data']
> > Then I suspect the data will be sent from e1 up to the master engine and
> > then down to e0. How can I skip the intermediate step?
> Yup, that's what happens now. I know Min a little while ago whipped
> up something for engine-to-engine communication in zmq, but there's no
> high-level interface for that yet. This implements a simple 2d
> parallel wave solver, and it's the only example we have at this point
> going in this direction:
> Before we had punted on this problem altogether, b/c the overhead of
> Twisted was so high that for all sensible use cases needing this, the
> answer was always 'use mpi'. Now, with zmq that's not the case
> anymore, so we'll probably add some support for mpi-like
> communications using purely zmq and the ipython api, but that hasn't
> happened yet.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the IPython-User