[IPython-User] Scattering functions using IPython
Wed Aug 8 18:05:08 CDT 2012
> Message: 4
> Date: Wed, 8 Aug 2012 12:10:33 -0700
> From: MinRK <firstname.lastname@example.org>
> Subject: Re: [IPython-User] Scattering functions using IPython
> > Hi,
> > I'm trying to use IPython's scatter mechanism to distribute a large
> > number of dynamically generated functions, but I'm stymied by pickling
> > errors. Some furious googling about this issue reveals that pickling
> > functions is problematic, but is there anything in the IPython kit that
> > can make this work? It would be a lot of work to restructure my entire
> > codebase that relies heavily on these dynamically generated functions.
> > For reference, the functions themselves do not rely on global variables
> > in the namespace and are otherwise pretty self-contained.
> Are you using master? If so, try applying PR
> and see if it helps. The serialization is being redone a bit, and it is
> more conservative about inspecting arguments. 2246 fixes an issue where it
> was a little *too* conservative.
> We can handle simple functions, but most importantly not ones with
> closures. What error do you get?
Here's an example:
from IPython import parallel
rc = parallel.Client()
flist=[ eval('lambda x:sin(2*x-%3.3f)'% d) for d in linspace(0,1,10)]
PicklingError: Can't pickle <type 'function'>: attribute lookup
More information about the IPython-User