Also, your code works exactly as-is with <a href="https://github.com/ipython/ipython/pull/1990">this change</a> (the `view['f'] = f` step remains unnecessary).<div><br></div><div>-MinRK<br><br><div class="gmail_quote">
On Tue, Jun 19, 2012 at 7:54 PM, MinRK <span dir="ltr"><<a href="mailto:benjaminrk@gmail.com" target="_blank">benjaminrk@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote"><div><div class="h5">On Tue, Jun 19, 2012 at 7:14 PM, Robert Nishihara <span dir="ltr"><<a href="mailto:robertnishihara@gmail.com" target="_blank">robertnishihara@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<span>Hi,</span><div><font color="#222222" face="arial, sans-serif"><br></font></div><div><div>I have been running into a problem using map_sync in IPython.parallel (please tell me if this is the wrong place to ask this question). The script below works fine:</div>
<div><br></div><div><div> from IPython import parallel</div><div><br></div><div> def run_stuff():</div><div> def func(x):</div><div> return x</div><div> rc = parallel.Client(profile='sge')</div>
<div> dview = rc[:]</div><div> dview['func'] = func</div><div> print dview.map_sync(func, range(10))</div><div><br></div><div> run_stuff()</div></div><div><br></div><div>
The issue occurs when I want "func" to be parametrized by some arbitrary function "f". The script below is the simple example that illustrates the failure I am seeing</div><div><br></div><div>
<div> from IPython import parallel</div><div><br> def run_stuff(f):</div><div> rc = parallel.Client(profile='sge')</div><div> dview = rc[:]</div><div> dview['f'] = f</div>
<div> def func(x,f=f):</div><div> return f(x)</div><div> print dview.map_sync(func, range(10))</div><div><br></div><div> def g(x):</div><div> return x</div><div><br>
</div>
<div> run_stuff(g)</div></div><div><br></div><div>This script fails with the error:</div><div><br></div><div> AttributeError: 'module' object has no attribute 'g'.</div>
<div><br></div><div>The name "g" is not accessible to the function "run_stuff". The two scripts are attached. Can anyone help?</div></div></blockquote><div><br></div></div></div><div>Here's a <a href="http://stackoverflow.com/questions/10857250/python-name-space-issues-with-ipython-parallel" target="_blank">detailed explanation</a> of how namespaces are resolved.</div>
<div><br></div><div>To breakdown your case:</div><div><br></div><div>def func(x, f=f)</div><div><br></div><div>creates a function with default value tied to the module locals (module being __main__). g is not defined remotely, so __main__.g doesn't resolve. We have some utilities that allow functions to be serialized in a way that works around this, but func.func_defaults is inspected and wrapped like func itself. If we added that, then this code would just work, I think.</div>
<div><br></div><div>If you avoid passing arguments via the func_defaults, and pass them as arguments explicitly, you avoid the problem:</div><div><br></div><div>def func(x,f):</div><div>...</div><div><br></div><div>view.map_sync(func, range(10), [f]*10)</div>
<div><br></div><div>Will avoid the issue.</div><div><br></div><div>There is no need to push f before passing it to map.</div><div><br></div><div>-MinRK</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<span><font color="#888888"><div>
<br></div><div>-Robert</div>
</font></span><br>_______________________________________________<br>
IPython-User mailing list<br>
<a href="mailto:IPython-User@scipy.org" target="_blank">IPython-User@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-user" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-user</a><br>
<br></blockquote></div><br>
</blockquote></div><br></div>