[IPython-User] parallel computations on a global variable
Tue Feb 12 17:23:19 CST 2013
On 12 Feb 2013 19:42, "MinRK" <email@example.com> wrote:
> On Mon, Feb 11, 2013 at 9:33 AM, Moritz Emanuel Beber <
>> Dear all,
>> I'm trying to parallalize the computation of all pairwise inner products
of the column vectors of a matrix. Unfortunately, I have some issues:
>> Since I want to avoid passing the large matrix many times, I pushed the
matrix to the engines, making it available to the namespace there. I can
access that matrix as a global just fine using a function defined in the
notebook, doing the same in a module fails... (Please have a look at the
two attached files.)
>> When I use an IPython.parallel.Reference to the global matrix and pass
that to the function, that works in the external module. Of course, this
means I have to pass a large list containing the reference to the map
> This is the most common confusion when working with IPython.parallel.
> The gist: interactively defined functions and objects are in the
`__main__` module, so `globals()` in those scopes points to the same
> When you move something to a module, you change what `globals()` points
to. You can use the @interactive decorator (in IPython.parallel)
> to make functions defined in modules behave as if they were interactively
defined (i.e. globals will be __main__ rather than the defining module).
Awesome, thank you for the explanation. I figured it must be a namespace
issue since the variable wasn't in the globals but I didn't know about the
> So basically, in your code all you need to add is:
> from IPython.parallel import require, interactive
> def parallel_inner(i, j):
> Also note that to decorate with require, you need the `@`, which is
missing in your code.
> More detail here and here.
A great reminder to start checking SO for IPython questions more often.
>> Insights and advice is appreciated,
>> IPython-User mailing list
> IPython-User mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the IPython-User