[IPython-User] Running multiple ipclusters on remote cluster w/ Sun Grid Engine.
Wed Sep 14 14:12:56 CDT 2011
On Wed, Sep 14, 2011 at 11:13, Fernando Perez <firstname.lastname@example.org> wrote:
> Hi Dharhas,
> On Wed, Sep 14, 2011 at 6:59 AM, Dharhas Pothina
> <Dharhas.Pothina@twdb.state.tx.us> wrote:
> > I ended up writing a script that connected to the cluster and made a copy
> > an already created profile with a new unique name, started ipcluster,
> > till the json file was created and then retrieved the json file for use
> in a
> > local client, runs my script and then cleans up afterwards.
> > This seems to be working fairly well except when the local script exits
> > because of an error. In that case, I need to log in and stop the engines,
> > clean up files etc manually.
> OK. We probably should remove the assumption of a 1 to 1 mapping
> between profiles and running clusters, but that will require a fair
> bit of reorganization of code that uses that assumption, so I'm glad
> you found a solution for now.
Yes, it's a pretty big deal that the only thing engines and clients need to
know to connect to a cluster is the profile name. That is lost entirely if
we allow multiple clusters with a single profile, since profile name becomes
ambiguous. We would then need to add a second layer of specification for
which controller to use within a given profile, e.g.:
ipengine --profile=mysge --controller-id=12345
I think I could add support for exactly this without much code change at
Feature Request opened on GitHub:
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the IPython-User