[IPython-User] Parallel IPython -- too many open files

MinRK benjaminrk@gmail....
Sun Jan 8 13:27:54 CST 2012


On Sun, Jan 8, 2012 at 11:13, Fernando Perez <fperez.net@gmail.com> wrote:

> Hey Min,
>
> On Sat, Jan 7, 2012 at 11:27 PM, Erik Petigura <eptune@gmail.com> wrote:
> > I think this is the issue.  I added
> >
> >   rc.close()
> >
> > and the code ran to completion.
>
> perhaps we should add a __del__ method to the clients that calls
> close?  While python destructors aren't guaranteed to be called at
> shutdown and therefore should not be used to do things like remove
> external files, they do get called on object destruction in normal
> execution.  Since the rc in Erik's case would be destroyed at function
> exit b/c it goes out of scope, then having __del__ would have helped
> here.
>

> Do you see any reason not to?
>

Client *does* close on
del<https://github.com/ipython/ipython/blob/master/IPython/parallel/client/client.py#L388>.
 The problem is that somehow not every last reference is cleaned up.  Note
that *every* IPython.parallel  object ultimately has a reference to the
Client, even AsyncResult objects, paralllel functions, views, etc.  So `del
rc` almost never actually results in `rc.__del__()`.  Also note that
Client.close() on __del__ should also be redundant, as Socket.__del__
closes itself, and cleaning up the client should cleanup references to the
socket at the same time.

That said, I can't see the mechanism in the example code for a reference
sticking around.  The actual root cause of *that* might be a bug, though
not one I understand.

-MinRK


>
> f
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20120108/5870ccec/attachment.html 


More information about the IPython-User mailing list