[IPython-User] ipython parallelism and closures
Mon Apr 23 15:15:04 CDT 2012
On Mon, Apr 23, 2012 at 03:24, Andrew Jaffe <firstname.lastname@example.org> wrote:
> Dear all,
> I have what seems to be a very simply parallelizable task:
> ghdirs = [ './y/_x_%d/' % isim for isim in range(*nsim) ]
> gh = map(lambda ghd:
> GHcalc.getGH(freq=freq, num=num, nc=nc, GHdir=ghd,
> verbose=False, no_offset_correction=True),
> where the variables freq, num and nc are defined prior to this block.
> Because of the presence of these variables, this is a closure, which is
> unpicklable, and therefore cannot be sent to ipython clients, hence I
> get the message "ValueError: Sorry, cannot pickle code objects with
> closures" when I replace map with dview.map_sync.
> I am sure there must be a trivial way to parallize this... Can anyone help?
You just have to pass those values as arguments to your map function. They
can even be passed as default kwargs in your lambda:
ghdirs = [ './y/_x_%d/' % isim for isim in range(*nsim) ]
gh = map(lambda ghd, freq=freq, num=num, nc=nc:
GHcalc.getGH(freq=freq, num=num, nc=nc, GHdir=ghd,
> IPython-User mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the IPython-User