<br><br><div class="gmail_quote">On Sat, Jan 7, 2012 at 21:37, Fernando Perez <span dir="ltr"><<a href="mailto:fperez.net@gmail.com">fperez.net@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Olivier,<br>
<div class="im"><br>
On Fri, Jan 6, 2012 at 9:58 PM, Olivier Grisel <<a href="mailto:olivier.grisel@ensta.org">olivier.grisel@ensta.org</a>> wrote:<br>
> Very interesting thread, thanks for sharing. Let me share some of my<br>
> related use cases with to feed the IPython.parallel community with<br>
> some additional use cases you might not be aware of.<br>
<br>
</div>[...]<br>
<br>
Thanks a lot for this explanation and the link. I think I must have<br>
missed something, because it seems to me that we already offer that:<br>
<br>
In [26]: from IPython.parallel import Client<br>
...: v = Client()[:]<br>
<br>
In [27]: x = rand(len(v.targets))<br>
...: v.scatter('x', x)<br>
Out[27]: <AsyncResult: scatter><br>
<br>
In [29]: v['x']<br>
Out[29]:<br>
[array([ 0.55751144]),<br>
array([ 0.51093798]),<br>
array([ 0.03574132]),<br>
array([ 0.31362035])]<br>
<br>
In [30]: v['x_sum'] = sum(v['x'])<br>
...: v['x_sum']<br>
Out[30]:<br>
[1.4178110846551577,<br>
1.4178110846551577,<br>
1.4178110846551577,<br>
1.4178110846551577]<br>
<br>
<br>
No? That last line performs what the all_reduce operation is defined<br>
as: it pulls 'x' from each node, does the reduction, and sends back<br>
the global reduction to all nodes now named x_sum.<br>
<br>
Can you clarify what I'm not understanding?<br></blockquote><div><br></div><div>Fernando,</div><div><br></div><div>The problem is that gathering everything to the client prior to calling reduce won't work for the cases where the whole parallel problem doesn't fit in memory on one node, so you have to do a *parallel* reduce, which can be implemented without requiring more than two parts of the problem on a node at any given time.</div>
<div><br></div><div>Olivier,</div><div><br></div><div>Thanks for this data point, I have been thinking about what a 'zeromq-style' parallel reduce might look like. Obviously, you can implement one with all:all addressed messages (ROUTERs everywhere), but it would be nice to let 0MQ do more of the legwork. And the difference between reduce and allreduce is trivial, if I understand correctly - just publish the final reduced result.</div>
<div><br></div><div>-MinRK</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Thanks!<br>
<span class="HOEnZb"><font color="#888888"><br>
f<br>
</font></span></blockquote></div><br>