[IPython-User] Scattering functions using IPython
Wed Aug 8 18:27:39 CDT 2012
On Wed, Aug 8, 2012 at 4:05 PM, reckoner <firstname.lastname@example.org> wrote:
> > Message: 4
> > Date: Wed, 8 Aug 2012 12:10:33 -0700
> > From: MinRK <email@example.com>
> > Subject: Re: [IPython-User] Scattering functions using IPython
> > parallel
> > > Hi,
> > >
> > > I'm trying to use IPython's scatter mechanism to distribute a large
> > > number of dynamically generated functions, but I'm stymied by pickling
> > > errors. Some furious googling about this issue reveals that pickling
> > > functions is problematic, but is there anything in the IPython kit
> > > can make this work? It would be a lot of work to restructure my entire
> > > codebase that relies heavily on these dynamically generated functions.
> > > For reference, the functions themselves do not rely on global
> > > in the namespace and are otherwise pretty self-contained.
> > >
> > Are you using master? If so, try applying PR
> > 2246<https://github.com/ipython/ipython/pull/2246>,
> > and see if it helps. The serialization is being redone a bit, and it is
> > more conservative about inspecting arguments. 2246 fixes an issue
> where it
> > was a little *too* conservative.
> > We can handle simple functions, but most importantly not ones with
> > closures. What error do you get?
> Here's an example:
> from IPython import parallel
> rc = parallel.Client()
> flist=[ eval('lambda x:sin(2*x-%3.3f)'% d) for d in linspace(0,1,10)]
> PicklingError: Can't pickle <type 'function'>: attribute lookup
> __builtin__.function failed
I can confirm that the above-mentioned PR #2246 fixes this issue
You can even avoid the eval, with:
flist = [ lambda x,offset=d: sin(2*x - offset) for d in linspace(0,1,10) ]
It should also work just fine with 0.13, though I haven't checked.
> IPython-User mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the IPython-User