<html>
<head>
</head>
<body style="font-style: normal; font-weight: normal; line-height: normal; margin-bottom: 1px; font-family: Lucida Grande; margin-top: 4px; font-size: 12pt; font-variant: normal; margin-left: 4px; margin-right: 4px">
<br>
<p style="margin-top: 0; margin-bottom: 0">
<font size="3" face="Lucida Grande">One more related question. I've got things working as described below and it looks like each time I start ipcluster it overwrites the ipcontroller_client.json and the ipcontroller_engine.json file. Does this mean I can only have one ipcluster running? Or if I start the ipcluster and then copy the json file to the clients that require them can I then start another ipcluster job and use the new json file for the new clients.</font> </p>
<br>
<p style="margin-top: 0; margin-bottom: 0">
<font size="3" face="Lucida Grande">thanks,</font> </p>
<br>
<p style="margin-top: 0; margin-bottom: 0">
<font size="3" face="Lucida Grande">- dharhas</font><br><br>>>> MinRK <benjaminrk@gmail.com> 8/24/2011 5:07 PM >>><br>On Wed, Aug 24, 2011 at 15:05, Dharhas Pothina<br><Dharhas.Pothina@twdb.state.tx.us> wrote:<br>><br>> I was able to start the engines and they were submitted to the queue<br>> properly but I do not have a json file in the corresponding security folder.<br>> Do I need to do something to generate it.<br><br>The JSON file is written by ipcontroller, so it will only show up<br>after the controller has started.<br><br>><br>> - dharhas<br>><br>>>>> MinRK <benjaminrk@gmail.com> 8/24/2011 4:44 PM >>><br>> On a login node on the cluster:<br>><br>> # create profile with default parallel config files, called sge<br>> [login] $> ipython profile create sge --parallel<br>><br>> edit IPYTHON_DIR/profile_sge/ipcontroller_config.py, adding the line:<br>><br>> c.HubFactory.ip = '0.0.0.0'<br>><br>> to instruct the controller to listen on all interfaces.<br>><br>> Edit IPYTHON_DIR/profile_sge/ipcluster_config.py, adding the line:<br>><br>> c.IPClusterEngines.engine_launcher_class = 'SGEEngineSetLauncher'<br>> c.IPClusterStart.controller_launcher_class = 'SGEControllerLauncher'<br>><br>> # optional: specify a queue for all:<br>> c.SGELauncher.queue = 'short'<br>> To instruct ipcluster to use SGE to launch the engines and the controller<br>><br>> At this point, you can start 10 engines and a controller with:<br>><br>> [login] $> ipcluster start -n 10 --profile=sge<br>><br>> Now the only file you will need to connect to the cluster will be in:<br>><br>> IPYTHON_DIR/profile_sge/security/ipcontroller_client.json<br>><br>> Just move that file around, and you will be able to connect clients.<br>> To connect from a laptop, you will probably need to specify a login<br>> node as the ssh server when you do:<br>><br>> from IPython import parallel<br>><br>> rc = parallel.Client('/path/to/ipcontroller_client.json',<br>> sshserver='you@login.mycluster.etc')<br>><br>> -MinRK<br>><br>><br>> On Wed, Aug 24, 2011 at 13:18, Dharhas Pothina<br>> <Dharhas.Pothina@twdb.state.tx.us> wrote:<br>>> Hi All,<br>>><br>>> We have managed to parallelize one of our spatial interpolation scripts<br>>> very<br>>> easily with the new ipython parallel. Thanks for developing such a great<br>>> tool, it was fairly easy to get working. Now we are trying to set things<br>>> up<br>>> to run on our internal cluster and I'm having difficulties understanding<br>>> how<br>>> to configure things.<br>>><br>>> What I would like to do is have ipython running on a local machine<br>>> (windows<br>>> & linux) connect to the cluster, request some nodes through SGE and run<br>>> the<br>>> computation. I'm not quite getting what goes where from the documentation.<br>>><br>>> I think I understood the PBS example but I'm still not understanding where<br>>> I<br>>> would put the connection information to log into the cluster. I would<br>>> really<br>>> appreciate a step by step of what files need to be where and any example<br>>> config files for an SGE setup.<br>>><br>>> thanks,<br>>><br>>> - dharhas<br>>><br>>><br>>><br>>><br>>><br>>> _______________________________________________<br>>> IPython-User mailing list<br>>> IPython-User@scipy.org<br>>> <a href="http://mail.scipy.org/mailman/listinfo/ipython-user">http://mail.scipy.org/mailman/listinfo/ipython-user</a><br>>><br>>><br>> _______________________________________________<br>> IPython-User mailing list<br>> IPython-User@scipy.org<br>> <a href="http://mail.scipy.org/mailman/listinfo/ipython-user">http://mail.scipy.org/mailman/listinfo/ipython-user</a><br>><br>> _______________________________________________<br>> IPython-User mailing list<br>> IPython-User@scipy.org<br>> <a href="http://mail.scipy.org/mailman/listinfo/ipython-user">http://mail.scipy.org/mailman/listinfo/ipython-user</a><br>><br>><br>
</p>
</body>
</html>