[IPython-user] pushing/pulling objects between engines

Brian Granger ellisonbg.net@gmail....
Tue Oct 2 16:16:37 CDT 2007


What Fernando said is true.  It is worth noting that one of the key
reasons that we don't have a custom way of sending objects directly
between engines is that MPI is the best way of doing this (it is much
better than anything we would come up with).

Thus, if you really need to do this, I would use mpi4py in conjunction
with ipython1.  Mpi4py is a absolutely fantastic project and ipython1
has full support for it.  For instance, if you have mpi4py installed,
you can start the engines using

mpirun -n 4 ipengine --mpi=mpi4py

>From that point on, you can send python command to the engines (using
execute/executeAll) that contain calls to mpi4py.  The benefit of this
approach is that if your cluster has a fast interconnect (like
infiniband) it will be used to send the objects between the engines.

If you go this route, let us know if you run into problems.

Cheers,

Brian

On 10/2/07, Jonathan Taylor <jonathan.taylor@stanford.edu> wrote:
> Is it possible to send information between two engines using the
> RemoteController?
> I can imagine wanting to do something like the following between two
> engines i and j
>
> a_i, a_j = a_j, a_i
>
> (not proper python syntax of course).
>
> I didn't see an example of this in the tutorial.... other than
>
>  >>> a_i, a_j = rc.pull(i, 'a'), rc.pull(j, 'a')
>  >>> rc.push(j, 'a'=a_i); rc.push(i, 'a'=a_j)
>
> Any suggestions?
>
> Thanks,
>
> Jonathan
>
> _______________________________________________
> IPython-user mailing list
> IPython-user@scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-user
>


More information about the IPython-user mailing list