[IPython-User] Running multiple ipclusters on remote cluster w/ Sun Grid Engine.

Dharhas Pothina Dharhas.Pothina@twdb.state.tx...
Wed Sep 14 08:59:48 CDT 2011


I ended up writing a script that connected to the cluster and made a copy of an already created profile with a new unique name, started ipcluster, waited till the json file was created and then retrieved the json file for use in a local client, runs my script and then cleans up afterwards. 

This seems to be working fairly well except when the local script exits because of an error. In that case, I need to log in and stop the engines, clean up files etc manually. 

- dharhas

>>> MinRK <benjaminrk@gmail.com> 9/12/2011 4:46 PM >>>

On Mon, Sep 12, 2011 at 14:35, Fernando Perez <fperez.net@gmail.com> wrote:

Hi Dharhas,

On Fri, Sep 2, 2011 at 6:38 AM, Dharhas Pothina

<Dharhas.Pothina@twdb.state.tx.us> wrote:

> One more related question. I've got things working as described below and it
> looks like each time I start ipcluster it overwrites the
> ipcontroller_client.json and the ipcontroller_engine.json file. Does this
> mean I can only have one ipcluster running? 

Currently there is an expectation that controllers are singletons *per profile*. You can start as many controllers as you like, as long as they are using different profiles. 

Or if I start the ipcluster and
> then copy the json file to the clients that require them can I then start
> another ipcluster job and use the new json file for the new clients.

If you pass the --reuse flag directly to the ipcontroller script, it
will reuse a connection file. Let us know if this helps...  



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20110914/1163f7ac/attachment.html 

More information about the IPython-User mailing list