[IPython-dev] Before a patch for LSF support

Matthieu Brucher matthieu.brucher@gmail....
Mon Aug 10 13:22:18 CDT 2009


2009/8/10 Brian Granger <ellisonbg.net@gmail.com>:
>
>> I've tested again and it works with local and ssh (ssh on a remote
>> cluster that has a different $HOME).
>> I still have issues to get the log files from LSF, although the jobs
>> are submitted correctly (BTW, is there an explanation of how PBS
>> engine is set up?)
>
> The only real explanation of how the PBS ipcluster works is the code
> itself.  Basically, it goes through the following sequence:
>
> 1.  Start ipcontroller on the same node that ipcluster is being run.
> 2.  Construct a PBS batch script that calls ipengine on each node and then
> submit the batch script.
> 3.  Get the job ID from the output of qsub.
>
> You should be able to get LSF working by:
>
> 1.  Changing the submission syntax.

This is already done (not complicated, two names to change!)

> 2.  Writing your own template for the batch script (test the batch script
> without ipcluster first!)

Should be OK, I wrote several LSF scripts.

> 3.  Write your own regexp for parsing the job ID
> 4.  Possibly add logic for copying the furl files around or for setting the
> command line options to point to them is they are on different locations.

This may be the only thing that I couldn't check.

> Keep us posted as it would be great to have LSF support.

Of course I will! I will also try to get a patch against the trunk
(for th emoment, I only test against 0.10 due to proxy issues on my
side).

Cheers,

Matthieu
-- 
Information System Engineer, Ph.D.
Website: http://matthieu-brucher.developpez.com/
Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92
LinkedIn: http://www.linkedin.com/in/matthieubrucher


More information about the IPython-dev mailing list