[IPython-User] Limitations (?) of ipython SGE support

Brian Granger ellisonbg@gmail....
Sun Jan 16 20:59:14 CST 2011


On Thu, Jan 13, 2011 at 7:41 AM, Chris Filo Gorgolewski
<chris.gorgolewski@gmail.com> wrote:
> Hi,
> I have recently played with ipython on our SGE cluster. I was
> surprised to discover that ipython does not use qsub to submit every
> job, but submits prespecified number of ipengines as jobs. Those I
> presume run indefinitely and accept ipython tasks. This setup seem to
> have two major drawbacks:

Yes, this is a correct description of what happens.

> 1) my cluster have nodes with different max job time. Depending what
> you specify in the qsub option the job gets send to different node.
> The limit is 48h. This means that after 48h (assuming that I use a
> custom submit script with this option) all of my engines will be
> killed and ipython will stop receiving jobs?

Yes, that is right.

> In other words I cannot
> run a set of jobs that would run longer than two days using ipython?

Yep, there is no way of getting around the limitations/constraints of
the queues.

> Additionally if I decide to specify max job time 48h I will most
> likely wait longer for the appropriate nodes to become free which is
> not really necessary when my atomic jobs run much faster.

Yep, such is life on shared clusters with batch system :(

What about just firing up an EC2 cluster using startcluster?

> 2) I need to specify how many engines I want to use. Assuming i want
> my set of jobs to be done as quickly as possible I should specify a
> number that would be bigger than the number of available nodes. This
> means that in many situations I will spawn way too many ipengines that
> will just sit there doing nothing. This solution seems to lack
> scalability.
> Or maybe I am using ipython/SGE in a wrong way?

>From what we have said, I think you are using ipython/sge in the right
manner, you are just running into the fact that batch systems are not
setup for truly interactive usage.



> Best regards,
> Chris
> _______________________________________________
> IPython-User mailing list
> IPython-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-user

Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo

More information about the IPython-User mailing list