[IPython-User] Parallel question: Sending data directly between engines
Wed Jan 25 15:57:34 CST 2012
I see in PR 1295 you're pickling all transmitted objects. Is there a
cleaner/faster way of doing this for arrays?
Using cPickle on a 1000 by 1000 array of float32s takes about a second on
On Wed, Jan 25, 2012 at 3:34 PM, Matthew Rocklin <email@example.com> wrote:
> I'm going to rehijack this thread back to the simple peer-peer case.
> I was playing with the interengine example with numpy arrays and running
> into issues. It looks like its sending the array's data but none of the
> object data.
> In : import numpy as np
> In : run interengine.py
> In : A, B = rc, rc
> In : A['data'] = np.ones((2,2))
> In : send(rc, 0, 1, 'data')
> In : A['data']
> array([[ 1., 1.],
> [ 1., 1.]])
> In : B['data']
> Is there a clean and fast way to make B's data variable a numpy array? I
> suspect this is handled already in the high-level interface. I wasn't able
> to locate this bit of code. Is there somewhere in particular I should be
> looking for this kind of thing?
> On Thu, Jan 19, 2012 at 7:38 PM, Fernando Perez <firstname.lastname@example.org>wrote:
>> On Thu, Jan 19, 2012 at 4:50 PM, MinRK <email@example.com> wrote:
>> > simple binary-tree engine interconnect example:
>> > https://github.com/ipython/ipython/pull/1295
>> Thanks! Olivier, if you have any feedback on this one, we'd appreciate
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the IPython-User