[IPython-User] Problem with load_balanced_view()

MinRK benjaminrk@gmail....
Fri Jun 15 14:56:28 CDT 2012

On Fri, Jun 15, 2012 at 7:54 AM, Neal Becker <ndbecker2@gmail.com> wrote:

> What am I doing wrong here?  Tried to call 'run_line' directly, but I get
> 'run_line' not defined.  So I tried again with decorator, but no diff:
> from my_script import run_line
> from IPython.parallel import Client
> rc = Client()
> lview = rc.load_balanced_view()
> if __name__ == '__main__':
> [ build list of tests...]
>    @lview.parallel()
>    def run_it (cmdline):
>        return run_line (cmdline)
>    parallel_result = lview.map(lambda t: run_it(cmdline=t), tests)
> CompositeError: one or more exceptions from call to method: <lambda>
> [4:apply]: NameError: global name 'run_it' is not defined
> [1:apply]: NameError: global name 'run_it' is not defined
> [0:apply]: NameError: global name 'run_it' is not defined
> [2:apply]: NameError: global name 'run_it' is not defined
> [7:apply]: NameError: global name 'run_it' is not defined
> [3:apply]: NameError: global name 'run_it' is not defined

The parallel decorator makes a function that will run remotely, so you
would do:

def run_it(cmdline):
    # import might be needed here for namespace reasons
    from my_script import run_line


The decorated run_it function *itself* executes remotely, so you wouldn't
want to pass it to apply/map.

If you want to use view.apply/map, then you don't need to make
parallel/remote functions with the decorators.

Also note that name resolution inside interactively defined functions is
based on the *engine* globals, not the client's,
so the following:

import string

def fetch_lower():
    return string.lowercase


will raise NameError on string

There are (at least) four ways to deal with imports on engines:

You can sync local/remote imports (this works in most cases, but not
'import foo as bar'):

with view.sync_imports():
    import string

just execute the remote import statement:

view.execute("import string")

decorate functions with `@require` for the modules they need:

from IPython.parallel import require

@require('string') # can pass name *or* module
def fetch_lower():
    return string.lowercase

or (this is the one I used most often) just put imports inside your
interactively defined functions:

def fetch_lower():
    import string
    return string.lowercase


> _______________________________________________
> IPython-User mailing list
> IPython-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-user
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20120615/fc0a77ba/attachment-0001.html 

More information about the IPython-User mailing list