[IPython-User] Sharing data between engines and using modules

[seshu yamajala]

On Aug 16, 2012, at 9:19 AM, seshu yamajala wrote:

> On Aug 15, 2012, at 9:09 PM, seshu yamajala wrote:
> 
>> 
>> On Aug 15, 2012, at 8:39 PM, MinRK wrote:
>> 
>>> 
>>> 
>>> On Wed, Aug 15, 2012 at 4:29 PM, seshu yamajala <syamajala@gmail.com> wrote:
>>> Hello,
>>> 
>>> I was wondering what the proper way to share data between engines is when your functions are defined in a module which is imported on all the engines. I have a function in my module which computes a dict and pushes that dict to all the engines, but when i try to access that dict from another function in the module it says that the global name is undefined. Looking at the code for push, I saw that it was decorated with @interactive. Is there another way to do this?
>>> 
>>> It all depends on what you want to do.  The user_ns is effectively __main__.__dict__, so when you push/pull that is the namespace you are working with.
>>> 
>>> The @interactive decorator only makes the serialization code think that a function defined in a module is actually defined in the interactive namespace.  This means that when it is unpacked on the engine, its `globals` will refer to the user namespace rather than the module namespace.
>>> 
>>> A more thorough explanation.
>>> 
>>> Personally, I don't find global state of modules terribly attractive, so I prefer to use parallel.Reference(name) objects as arguments to a largely functional API, e.g.
>>> 
>>> def foo(x):
>>>     return x * x
>>> 
>>> A = numpy.ones(128)
>>> rc[:]['A'] = A
>>> 
>>> rA = parallel.Reference('A')
>>> ar = view.apply_async(foo, rA)
>>>  
>>> -MinRK
>>> 
>> 
>> Reference works for me. Thanks!
> 
> What if I want to call a method of an object that is Referenced on an engine? getObject() doesn't seem to work.



My mistake. In this case I was using Reference incorrectly. I should have passed a Reference to the function I wanted to call the object's method in, instead of trying to use Reference in the function itself. Everything is well now. 

Thanks,
Seshu


> Thanks,
> Seshu
> 
>> 
>> Seshu
>> 
>>> 
>>> Thanks,
>>> Seshu Yamajala
>>> 
>>> _______________________________________________
>>> IPython-User mailing list
>>> IPython-User@scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-user
>>> 
>>> _______________________________________________
>>> IPython-User mailing list
>>> IPython-User@scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-user
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20120816/17b8b922/attachment.html