[IPython-User] Sharing data between engines and using modules

[seshu yamajala]

On Aug 15, 2012, at 9:09 PM, seshu yamajala wrote:

> 
> On Aug 15, 2012, at 8:39 PM, MinRK wrote:
> 
>> 
>> 
>> On Wed, Aug 15, 2012 at 4:29 PM, seshu yamajala <syamajala@gmail.com> wrote:
>> Hello,
>> 
>> I was wondering what the proper way to share data between engines is when your functions are defined in a module which is imported on all the engines. I have a function in my module which computes a dict and pushes that dict to all the engines, but when i try to access that dict from another function in the module it says that the global name is undefined. Looking at the code for push, I saw that it was decorated with @interactive. Is there another way to do this?
>> 
>> It all depends on what you want to do.  The user_ns is effectively __main__.__dict__, so when you push/pull that is the namespace you are working with.
>> 
>> The @interactive decorator only makes the serialization code think that a function defined in a module is actually defined in the interactive namespace.  This means that when it is unpacked on the engine, its `globals` will refer to the user namespace rather than the module namespace.
>> 
>> A more thorough explanation.
>> 
>> Personally, I don't find global state of modules terribly attractive, so I prefer to use parallel.Reference(name) objects as arguments to a largely functional API, e.g.
>> 
>> def foo(x):
>>     return x * x
>> 
>> A = numpy.ones(128)
>> rc[:]['A'] = A
>> 
>> rA = parallel.Reference('A')
>> ar = view.apply_async(foo, rA)
>>  
>> -MinRK
>> 
> 
> Reference works for me. Thanks!

What if I want to call a method of an object that is Referenced on an engine? getObject() doesn't seem to work.

Thanks,
Seshu

> 
> Seshu
> 
>> 
>> Thanks,
>> Seshu Yamajala
>> 
>> _______________________________________________
>> IPython-User mailing list
>> IPython-User@scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-user
>> 
>> _______________________________________________
>> IPython-User mailing list
>> IPython-User@scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-user
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20120816/336e9f1e/attachment.html