[IPython-User] Parallel IPython -- too many open files

MinRK benjaminrk@gmail....
Sun Jan 8 13:45:54 CST 2012


On Sat, Jan 7, 2012 at 23:27, Erik Petigura <eptune@gmail.com> wrote:

> Hi, Min.
>
> On Jan 7, 2012, at 4:51 PM, Min RK wrote:
>
> > It would help a great deal to see your actual code, or better yet a
> minimal test case to cause the issue.
> >
>
> Here's the code that uses the IPython.parallel module.  This typically
> doesn't crash.  When I loop through it ~80 times, I get the too many files
> error:
>
>   def fitcandW(t,f,dL,par=False):
>       n = len(dL)
>       if par:
>           from IPython.parallel import Client
>           rc = Client()
>           lview = rc.load_balanced_view()
>           resL = lview.map(fitcand , n*[t] , n*[f] , dL, n*[False]
> ,block=True)
>       else:
>           resL = map(fitcand , n*[t] , n*[f] , dL)
>
>       return resL
>

I would recommend taking a view as an argument to your function, rather
than reestablishing the connection on every function call, which can be a
slow operation:

def fitcandW(t,f,dL,view=None):
    n = len(dL)
    if view is not None:
        resL = view.map(fitcand , n*[t] , n*[f] , dL, n*[False], block=True)
    else:
        resL = map(fitcand , n*[t] , n*[f] , dL)

    return resL

It would also let you use the same function with various views more easily
(load-balanced, direct, engine subsets, etc.).




>
>
> > Is it your client process that is running out of FDs?
> >
> > Are you perchance creating clients repeatedly?  If at all possible,
> continue to use the same client for the duration of the process.  Creating
> a client opens several socket connections to speak to the cluster, and if
> they are not properly cleaned up, then your fd count will obviously rise.
>  Deleting a client *should* close all its sockets, but that assumes you
> have actually cleared all references to it (not always an easy task,
> especially in an interactive IPython session).  Client.close() will
> explicitly close all of its sockets.
> >
>
> I think this is the issue.  I added
>
>   rc.close()
>
> and the code ran to completion.
>
> Thanks!
>
> Erik
>
>
> > -MinRK
> >
> > On Jan 7, 2012, at 16:00, Erik Petigura <eptune@gmail.com> wrote:
> >
> >> Dear IPython,
> >>
> >> I've been running code on 8 engines via:
> >>
> >>  ipcluster start --n=8
> >>
> >> The code ran for a while, but then it died with:
> >>
> >>  ERROR: Internal Python error in the inspect module.
> >>  Below is the traceback from this internal error.
> >>  Traceback (most recent call last):
> >>    File
> "/Library/Frameworks/EPD64.framework/Versions/7.1/lib/python2.7/site-packages/IPython/core/ultratb.py",
> line 750, in structured_traceback
> >>    File
> "/Library/Frameworks/EPD64.framework/Versions/7.1/lib/python2.7/site-packages/IPython/core/ultratb.py",
> line 232, in _fixed_getinnerframes
> >>    File
> "/Library/Frameworks/EPD64.framework/Versions/7.1/lib/python2.7/inspect.py",
> line 1041, in getinnerframes
> >>    File
> "/Library/Frameworks/EPD64.framework/Versions/7.1/lib/python2.7/inspect.py",
> line 1001, in getframeinfo
> >>    File
> "/Library/Frameworks/EPD64.framework/Versions/7.1/lib/python2.7/inspect.py",
> line 456, in getsourcefile
> >>    File
> "/Library/Frameworks/EPD64.framework/Versions/7.1/lib/python2.7/inspect.py",
> line 485, in getmodule
> >>    File
> "/Library/Frameworks/EPD64.framework/Versions/7.1/lib/python2.7/inspect.py",
> line 469, in getabsfile
> >>    File
> "/Library/Frameworks/EPD64.framework/Versions/7.1/lib/python2.7/posixpath.py",
> line 347, in abspath
> >> OSError: [Errno 24] Too many open files
> >>
> >> I increased the limits on the number of open files from 256 to:
> >>
> >>  [537][/Users/petigura]$ launchctl limit maxfiles
> >>       maxfiles    1000000        1000000
> >>
> >> My code ran longer but died.  Is IPython opening a bunch of files when
> it manages its engines?
> >>
> >> Thanks in advance for any help.
> >>
> >> Erik
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> IPython-User mailing list
> >> IPython-User@scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-user
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20120108/04ec8d05/attachment-0001.html 


More information about the IPython-User mailing list