[IPython-User] Sharing data between engines and using modules

MinRK benjaminrk@gmail....
Wed Aug 15 19:39:21 CDT 2012


On Wed, Aug 15, 2012 at 4:29 PM, seshu yamajala <syamajala@gmail.com> wrote:

> Hello,
>
> I was wondering what the proper way to share data between engines is when
> your functions are defined in a module which is imported on all the
> engines. I have a function in my module which computes a dict and pushes
> that dict to all the engines, but when i try to access that dict from
> another function in the module it says that the global name is undefined.
> Looking at the code for push, I saw that it was decorated with
> @interactive. Is there another way to do this?
>

It all depends on what you want to do.  The user_ns is effectively
__main__.__dict__, so when you push/pull that is the namespace you are
working with.

The @interactive decorator only makes the serialization code think that a
function defined in a module is actually defined in the interactive
namespace.  This means that when it is unpacked on the engine, its
`globals` will refer to the user namespace rather than the module namespace.

A more thorough
explanation<http://stackoverflow.com/questions/10857250/python-name-space-issues-with-ipython-parallel>
.

Personally, I don't find global state of modules terribly attractive, so I
prefer to use parallel.Reference(name) objects as arguments to a largely
functional API, e.g.

def foo(x):
    return x * x

A = numpy.ones(128)
rc[:]['A'] = A

rA = parallel.Reference('A')
ar = view.apply_async(foo, rA)

-MinRK


> Thanks,
> Seshu Yamajala
>
> _______________________________________________
> IPython-User mailing list
> IPython-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20120815/97119107/attachment.html 


More information about the IPython-User mailing list