<br><br><div class="gmail_quote">On Wed, Aug 8, 2012 at 4:05 PM, reckoner <span dir="ltr"><<a href="mailto:reckoner@gmail.com" target="_blank">reckoner@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
> Message: 4<br>
> Date: Wed, 8 Aug 2012 12:10:33 -0700<br>
> From: MinRK <<a href="mailto:benjaminrk@gmail.com">benjaminrk@gmail.com</a>><br>
> Subject: Re: [IPython-User] Scattering functions using IPython<br>
> parallel<br>
><br>
> > Hi,<br>
> ><br>
> > I'm trying to use IPython's scatter mechanism to distribute a large<br>
> > number of dynamically generated functions, but I'm stymied by pickling<br>
> > errors. Some furious googling about this issue reveals that pickling<br>
> > functions is problematic, but is there anything in the IPython kit that<br>
> > can make this work? It would be a lot of work to restructure my entire<br>
> > codebase that relies heavily on these dynamically generated functions.<br>
> > For reference, the functions themselves do not rely on global variables<br>
> > in the namespace and are otherwise pretty self-contained.<br>
> ><br>
><br>
> Are you using master? If so, try applying PR<br>
> 2246<<a href="https://github.com/ipython/ipython/pull/2246" target="_blank">https://github.com/ipython/ipython/pull/2246</a>>,<br>
> and see if it helps. The serialization is being redone a bit, and it is<br>
> more conservative about inspecting arguments. 2246 fixes an issue where it<br>
> was a little *too* conservative.<br>
><br>
> We can handle simple functions, but most importantly not ones with<br>
> closures. What error do you get?<br>
<br>
Here's an example:<br>
<br>
<br>
from IPython import parallel<br>
rc = parallel.Client()<br>
dv=rc[:]<br>
flist=[ eval('lambda x:sin(2*x-%3.3f)'% d) for d in linspace(0,1,10)]<br>
dv.scatter('foo',flist)<br>
PicklingError: Can't pickle <type 'function'>: attribute lookup<br>
__builtin__.function failed<br></blockquote><div><br></div><div>I can confirm that the above-mentioned PR #2246 fixes this issue</div><div><br></div><div>You can even avoid the eval, with:</div><div><br></div><div>flist = [ lambda x,offset=d: sin(2*x - offset) for d in linspace(0,1,10) ]</div>
<div><br></div><div><br></div><div>It should also work just fine with 0.13, though I haven't checked.</div><div><br></div><div>-MinRK</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
Thanks!<br>
<br>
<br>
<br>
_______________________________________________<br>
IPython-User mailing list<br>
<a href="mailto:IPython-User@scipy.org">IPython-User@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-user" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-user</a><br>
</blockquote></div><br>