[IPython-User] ipcluster's ssh mode and logging

Giovanni Luca Ciampaglia ciampagg@usi...
Tue Mar 15 04:26:45 CDT 2011


On 14/03/2011 19:17, Vishal Vatsa wrote:

> Could you provide information on your ssh cluster env.
> I would hazard a guess that, stderr are in the logs of the engines.
>

I launch the cluster with:
> ipcluster ssh --clusterfile cluster_conf.py 2>&1 >cluster.log &


cluster_conf.py
---------------
> send_furl = True
> engines = {
>         'mate1' : 2,
>         'mate2' : 2,
>         'mate3' : 4,
>         'mate4' : 4,
> }


cluster.log
-----------
> 2011-03-14 14:45:41+0100 [-] Log opened.
> 2011-03-14 14:45:41+0100 [-] Process ['ipcontroller', '--logfile=/home/gciampag/.ipython/log/ipcontroller'] has started with pid=18208
> 2011-03-14 14:45:41+0100 [-] Waiting for controller to finish starting...
> 2011-03-14 14:45:42+0100 [-] Controller started
> 2011-03-14 14:45:42+0100 [-] Copying furl file: scp ~/.ipython/security/ipcontroller-engine.furl mate1:.ipython/security/
> 2011-03-14 14:45:42+0100 [-] Copying furl file: scp ~/.ipython/security/ipcontroller-engine.furl mate2:.ipython/security/
> 2011-03-14 14:45:42+0100 [-] Copying furl file: scp ~/.ipython/security/ipcontroller-engine.furl mate3:.ipython/security/
> 2011-03-14 14:45:42+0100 [-] Copying furl file: scp ~/.ipython/security/ipcontroller-engine.furl mate4:.ipython/security/
> 2011-03-14 14:45:42+0100 [-]
> 2011-03-14 14:45:42+0100 [-] Copying sshx: scp /tmp/gciampag-main-sshx.sh mate1:/tmp/gciampag-sshx.sh
> 2011-03-14 14:45:42+0100 [-]
> 2011-03-14 14:45:42+0100 [-] Copying sshx: scp /tmp/gciampag-main-sshx.sh mate4:/tmp/gciampag-sshx.sh
> 2011-03-14 14:45:42+0100 [-]
> 2011-03-14 14:45:42+0100 [-] Copying sshx: scp /tmp/gciampag-main-sshx.sh mate2:/tmp/gciampag-sshx.sh
> 2011-03-14 14:45:42+0100 [-]
> 2011-03-14 14:45:42+0100 [-] Copying sshx: scp /tmp/gciampag-main-sshx.sh mate3:/tmp/gciampag-sshx.sh
> 2011-03-14 14:45:42+0100 [-] about to start engines...
> 2011-03-14 14:45:42+0100 [-] Starting engines: ssh mate1 sh /tmp/gciampag-sshx.sh ipengine
> 2011-03-14 14:45:42+0100 [-] Starting engines: ssh mate1 sh /tmp/gciampag-sshx.sh ipengine
> 2011-03-14 14:45:42+0100 [-] about to start engines...
> 2011-03-14 14:45:42+0100 [-] Starting engines: ssh mate4 sh /tmp/gciampag-sshx.sh ipengine
> 2011-03-14 14:45:42+0100 [-] Starting engines: ssh mate4 sh /tmp/gciampag-sshx.sh ipengine
> 2011-03-14 14:45:42+0100 [-] Starting engines: ssh mate4 sh /tmp/gciampag-sshx.sh ipengine
> 2011-03-14 14:45:42+0100 [-] Starting engines: ssh mate4 sh /tmp/gciampag-sshx.sh ipengine
> 2011-03-14 14:45:42+0100 [-] about to start engines...
> 2011-03-14 14:45:42+0100 [-] Starting engines: ssh mate2 sh /tmp/gciampag-sshx.sh ipengine
> 2011-03-14 14:45:43+0100 [-] Starting engines: ssh mate2 sh /tmp/gciampag-sshx.sh ipengine
> 2011-03-14 14:45:43+0100 [-] about to start engines...
> 2011-03-14 14:45:43+0100 [-] Starting engines: ssh mate3 sh /tmp/gciampag-sshx.sh ipengine
> 2011-03-14 14:45:43+0100 [-] Starting engines: ssh mate3 sh /tmp/gciampag-sshx.sh ipengine
> 2011-03-14 14:45:43+0100 [-] Starting engines: ssh mate3 sh /tmp/gciampag-sshx.sh ipengine
> 2011-03-14 14:45:43+0100 [-] Starting engines: ssh mate3 sh /tmp/gciampag-sshx.sh ipengine
> 2011-03-15 00:34:44+0100 [-] Copying engine_killer: scp /tmp/gciampag-local-engine_killer.sh mate1:/tmp/gciampag-engine_killer.sh
> 2011-03-15 00:34:44+0100 [-] Copying engine_killer: scp /tmp/gciampag-local-engine_killer.sh mate2:/tmp/gciampag-engine_killer.sh
> 2011-03-15 00:34:44+0100 [-] Copying engine_killer: scp /tmp/gciampag-local-engine_killer.sh mate3:/tmp/gciampag-engine_killer.sh
> 2011-03-15 00:34:44+0100 [-] Copying engine_killer: scp /tmp/gciampag-local-engine_killer.sh mate4:/tmp/gciampag-engine_killer.sh
> 2011-03-15 00:34:45+0100 [-] Process ['ipcontroller', '--logfile=/home/gciampag/.ipython/log/ipcontroller'] has stopped with 0
> 2011-03-15 00:34:46+0100 [-] Main loop terminated.

I inspected ipcontroller18208.log and ipcontroller-18208-engine-info.log 
but there is no sign of the console output.

By contrast, please see in attachment the log I get when running 
ipcluster local with:

> ipcluster local -n $cpus 2>&1 >cluster.log &

Best,

-- 
Giovanni L. Ciampaglia
PhD Student
University of Lugano, MACS Lab
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cluster_local.log
Type: text/x-log
Size: 374214 bytes
Desc: not available
Url : http://mail.scipy.org/pipermail/ipython-user/attachments/20110315/fc134ec6/attachment-0001.bin 


More information about the IPython-User mailing list