<br><br><div class="gmail_quote">On Fri, Jun 15, 2012 at 7:54 AM, Neal Becker <span dir="ltr"><<a href="mailto:ndbecker2@gmail.com" target="_blank">ndbecker2@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
What am I doing wrong here? Tried to call 'run_line' directly, but I get<br>
'run_line' not defined. So I tried again with decorator, but no diff:<br>
<br>
from my_script import run_line<br>
<br>
from IPython.parallel import Client<br>
<br>
rc = Client()<br>
lview = rc.load_balanced_view()<br>
<br>
if __name__ == '__main__':<br>
<br>
[ build list of tests...]<br>
<br>
@lview.parallel()<br>
def run_it (cmdline):<br>
return run_line (cmdline)<br>
<br>
parallel_result = lview.map(lambda t: run_it(cmdline=t), tests)<br>
<br>
CompositeError: one or more exceptions from call to method: <lambda><br>
[4:apply]: NameError: global name 'run_it' is not defined<br>
[1:apply]: NameError: global name 'run_it' is not defined<br>
[0:apply]: NameError: global name 'run_it' is not defined<br>
[2:apply]: NameError: global name 'run_it' is not defined<br>
[7:apply]: NameError: global name 'run_it' is not defined<br>
[3:apply]: NameError: global name 'run_it' is not defined<br></blockquote><div><br></div><div><br></div><div>The parallel decorator makes a function that will run remotely, so you would do:</div><div><br></div><div>
@view.parallel()<br>def run_it(cmdline):</div><div> # import might be needed here for namespace reasons</div><div> from my_script import run_line</div><div> run_line(cmdline)</div><div><br></div><div>run_it.map(tests)</div>
<div><br></div><div><br></div><div>The decorated run_it function *itself* executes remotely, so you wouldn't want to pass it to apply/map.</div><div><br></div><div>If you want to use view.apply/map, then you don't need to make parallel/remote functions with the decorators.</div>
<div><br></div><div>Also note that name resolution inside interactively defined functions is based on the *engine* globals, not the client's,</div><div>so the following:</div><div><br></div><div>import string</div><div>
<br></div><div>def fetch_lower():</div><div> return string.lowercase</div><div><br></div><div>view.apply_sync(fetch_lower)</div><div><br></div><div>will raise NameError on string</div><div><br></div><div>There are (at least) four ways to deal with imports on engines:</div>
<div><br></div><div>You can sync local/remote imports (this works in most cases, but not 'import foo as bar'):</div><div><br></div><div>with view.sync_imports():</div><div> import string</div><div><br></div><div>
just execute the remote import statement:</div><div><br></div><div>view.execute("import string")</div><div><br></div><div>decorate functions with `@require` for the modules they need:</div><div><br></div><div>from IPython.parallel import require</div>
<div><br></div><div>@require('string') # can pass name *or* module</div><div>def fetch_lower():</div><div> return string.lowercase</div><div><br></div><div>or (this is the one I used most often) just put imports inside your interactively defined functions:</div>
<div><br></div><div>def fetch_lower():</div><div> import string</div><div> return string.lowercase</div><div><br></div><div><br></div><div>-MinRK</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
_______________________________________________<br>
IPython-User mailing list<br>
<a href="mailto:IPython-User@scipy.org">IPython-User@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-user" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-user</a><br>
</blockquote></div><br>