[IPython-User] ipython cluster engines and fd limit

MinRK benjaminrk@gmail....
Thu Sep 20 15:54:06 CDT 2012


How are you starting the engines (ipcluster, launcher config, etc.)?  What
is your system like (shared filesystem, ssh, nfs, etc.)?

It's possible there are issues with too many simultaneous connection
attempts, which could be addressed by adding  a delay between each engine
start (see various `delay` configurables in ipcluster_config.py, depending
on how you are starting your cluster).

-MinRK

On Thu, Sep 20, 2012 at 8:01 AM, M. Wimmer <wimmer.mike@gmail.com> wrote:

> Just a little update: In fact the problem I described does not seem to have
> anything to do with the file descriptor limit. In /proc/pid/fd I can see
> that
> there are less than 1024 files open. Actually, when I try to start so many
> engines that the fd limit is reached, one of the ipcontroller processes
> dies
> (without leaving an error message in the log)
>
> I tried to increase the logging messages by adding logging statements
> myself in
> the code, but I'm still far from making progress (it's actually quite hard
> to
> follow the ipython code logic, as there are many different levels of
> inheritation - this is not a criticism; I'm just asking for help in this
> matter)
>
> All the best,
>
> Michael
>
> _______________________________________________
> IPython-User mailing list
> IPython-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20120920/92c410d4/attachment-0001.html 


More information about the IPython-User mailing list