[IPython-User] Getting setup on a remote cluster w/ Sun Grid Engine.
Wed Aug 24 17:05:12 CDT 2011
I was able to start the engines and they were submitted to the queue properly but I do not have a json file in the corresponding security folder. Do I need to do something to generate it.
>>> MinRK <firstname.lastname@example.org> 8/24/2011 4:44 PM >>>
On a login node on the cluster:
# create profile with default parallel config files, called sge
[login] $> ipython profile create sge --parallel
edit IPYTHON_DIR/profile_sge/ipcontroller_config.py, adding the line:
c.HubFactory.ip = '0.0.0.0'
to instruct the controller to listen on all interfaces.
Edit IPYTHON_DIR/profile_sge/ipcluster_config.py, adding the line:
c.IPClusterEngines.engine_launcher_class = 'SGEEngineSetLauncher'
c.IPClusterStart.controller_launcher_class = 'SGEControllerLauncher'
# optional: specify a queue for all:
c.SGELauncher.queue = 'short'
To instruct ipcluster to use SGE to launch the engines and the controller
At this point, you can start 10 engines and a controller with:
[login] $> ipcluster start -n 10 --profile=sge
Now the only file you will need to connect to the cluster will be in:
Just move that file around, and you will be able to connect clients.
To connect from a laptop, you will probably need to specify a login
node as the ssh server when you do:
from IPython import parallel
rc = parallel.Client('/path/to/ipcontroller_client.json',
On Wed, Aug 24, 2011 at 13:18, Dharhas Pothina
> Hi All,
> We have managed to parallelize one of our spatial interpolation scripts very
> easily with the new ipython parallel. Thanks for developing such a great
> tool, it was fairly easy to get working. Now we are trying to set things up
> to run on our internal cluster and I'm having difficulties understanding how
> to configure things.
> What I would like to do is have ipython running on a local machine (windows
> & linux) connect to the cluster, request some nodes through SGE and run the
> computation. I'm not quite getting what goes where from the documentation.
> I think I understood the PBS example but I'm still not understanding where I
> would put the connection information to log into the cluster. I would really
> appreciate a step by step of what files need to be where and any example
> config files for an SGE setup.
> - dharhas
> IPython-User mailing list
IPython-User mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the IPython-User