[IPython-dev] Before a patch for LSF support

Brian Granger ellisonbg.net@gmail....
Fri Oct 30 17:46:52 CDT 2009


Matthieu,

I have spent the last month completely refactoring ipcluster.  This work
should make it easier to add support for LSF.  This work should be merged
into trunk in the next few weeks.  But, I think you are facing some unique
issues described below...

I'm trying to get ipython working with LSF. I did manage to submit the
> job and let it run on the nodes.
> The first issue I face is that $HOME is not the same location on the
> LSF nodes than on the main computer. This is an issue that should be
> tackled with the refactoring of ipython (adding the possibility of
> setting the path for the controller and a different pass for the
> engines, if I understood ipython correctly).
>

When is this an issue?  When finding the .ipython directory?  When importing
ipython (PYTHONPATH)?  I think this one can be solved without making any
changes
to IPython, but if we need to we can.


> Meanwhile, I've tried to launch ipcluster from an LSF node (I have an
> SSH access too for debugging purposes). So in this case, I don't have
> a $HOME problem. This time, it's the ipcontroller-mec.furl file that
> has a 127.0.0.1 as the controller IP. I've tried to replace it with
> the node name, but I also a Conenction Refused error in the engines
> logs when I try to create a MultiEngineClient.
>

Ahh, yes, this type of thing can be subtle to get working - especially on
clusters with multiple
tcp/ip interfaces.  With my refactored work, these things can easily be
configured.  There are two things to look at:

--engine-ip = set's the ip address (or hostname) that the controller will
listen on.  Default is probably fine.
--engine-location = set's the ip address (or hostname) that will appear in
the furl.  This is the ip address that
the engine will try to connect on.  I would try changing this one.


> I hope I summed up my issue properly.
>

Yes, with all the changes going on in trunk, this might take a while to get
figured out, but I think it is doable.

Cheers,

Brian


> Cheers,
>
> Matthieu
> --
> Information System Engineer, Ph.D.
> Website: http://matthieu-brucher.developpez.com/
> Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92
> LinkedIn: http://www.linkedin.com/in/matthieubrucher
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-dev/attachments/20091030/225451f0/attachment.html 


More information about the IPython-dev mailing list