[IPython-User] Parallel question: Sending data directly between engines
Sat Jan 7 23:37:29 CST 2012
On Fri, Jan 6, 2012 at 9:58 PM, Olivier Grisel <firstname.lastname@example.org> wrote:
> Very interesting thread, thanks for sharing. Let me share some of my
> related use cases with to feed the IPython.parallel community with
> some additional use cases you might not be aware of.
Thanks a lot for this explanation and the link. I think I must have
missed something, because it seems to me that we already offer that:
In : from IPython.parallel import Client
...: v = Client()[:]
In : x = rand(len(v.targets))
...: v.scatter('x', x)
Out: <AsyncResult: scatter>
In : v['x']
In : v['x_sum'] = sum(v['x'])
No? That last line performs what the all_reduce operation is defined
as: it pulls 'x' from each node, does the reduction, and sends back
the global reduction to all nodes now named x_sum.
Can you clarify what I'm not understanding?
More information about the IPython-User