[IPython-User] ipcluster ssh and remote port forwarding

Giovanni Luca Ciampaglia giovanni.luca.ciampaglia@usi...
Wed Sep 15 18:48:21 CDT 2010


On 16/09/2010 00:58, Brian Granger wrote:

>>
>> How do I use the same trick, but with ipcluster ssh? That is, how do I
>> tell to ipcluster which port the engines should connect to so that they
>> can talk to the ipcontroller?
>
> Assuming you are using IPython version 0.10, the ports are controlled
> by these flags:
>
>    --client-port CLIENT_PORT
>                          the port the controller will listen on for client
>                          connections
>    --engine-port ENGINE_PORT
>                          the port the controller will listen on for engine
>                          connections
>

Hi Brian,
Yes, I have 0.10. I tried using those options and reusing the furl files 
(-r).

ipcluster ssh tries to start the ipengine instances on host B but it 
appear to not succeed in that. However, once the ipcontroller has 
started, if I ssh host B and start manually the engines, they do 
connect. After removing all furl files I do:

> [gciampag@mate4 wikimodels_devel]$ ipcluster ssh --clusterfile cluster.py --engine-port=4000 --client-port=4001 -r
> 2010-09-16 01:40:01+0200 [-] Log opened.
> 2010-09-16 01:40:01+0200 [-] Process ['ipcontroller', '--logfile=/home/gciampag/.ipython/log/ipcontroller', '-r', '--client-port=4001', '--engine-port=4000'] has started with pid=3108
> 2010-09-16 01:40:01+0200 [-] Waiting for controller to finish starting...
> 2010-09-16 01:40:01+0200 [-] Controller started
> 2010-09-16 01:40:01+0200 [-] Copying furl file: scp ~/.ipython/security/ipcontroller-engine.furl mate1:.ipython/security/
> 2010-09-16 01:40:01+0200 [-] Copying furl file: scp ~/.ipython/security/ipcontroller-engine.furl mate2:.ipython/security/
> 2010-09-16 01:40:01+0200 [-] Copying furl file: scp ~/.ipython/security/ipcontroller-engine.furl mate3:.ipython/security/
> 2010-09-16 01:40:01+0200 [-] Copying furl file: scp ~/.ipython/security/ipcontroller-engine.furl mate4:.ipython/security/
> 2010-09-16 01:40:01+0200 [-] Copying furl file: scp ~/.ipython/security/ipcontroller-engine.furl research.inf.unisi.ch:.ipython/security/
> 2010-09-16 01:40:01+0200 [-]
> 2010-09-16 01:40:01+0200 [-] Copying sshx: scp /tmp/gciampag-main-sshx.sh mate2:/tmp/gciampag-sshx.sh
> 2010-09-16 01:40:01+0200 [-]
> 2010-09-16 01:40:01+0200 [-] Copying sshx: scp /tmp/gciampag-main-sshx.sh mate4:/tmp/gciampag-sshx.sh
> 2010-09-16 01:40:01+0200 [-]
> 2010-09-16 01:40:01+0200 [-] Copying sshx: scp /tmp/gciampag-main-sshx.sh mate1:/tmp/gciampag-sshx.sh
> 2010-09-16 01:40:01+0200 [-]
> 2010-09-16 01:40:01+0200 [-] Copying sshx: scp /tmp/gciampag-main-sshx.sh mate3:/tmp/gciampag-sshx.sh
> 2010-09-16 01:40:01+0200 [-]
> 2010-09-16 01:40:01+0200 [-] Copying sshx: scp /tmp/gciampag-main-sshx.sh research.inf.unisi.ch:/tmp/gciampag-sshx.sh
> 2010-09-16 01:40:02+0200 [-] about to start engines...
> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate4 sh /tmp/gciampag-sshx.sh ipengine
> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate4 sh /tmp/gciampag-sshx.sh ipengine
> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate4 sh /tmp/gciampag-sshx.sh ipengine
> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate4 sh /tmp/gciampag-sshx.sh ipengine
> 2010-09-16 01:40:02+0200 [-] about to start engines...
> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate3 sh /tmp/gciampag-sshx.sh ipengine
> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate3 sh /tmp/gciampag-sshx.sh ipengine
> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate3 sh /tmp/gciampag-sshx.sh ipengine
> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate3 sh /tmp/gciampag-sshx.sh ipengine
> 2010-09-16 01:40:02+0200 [-] about to start engines...
> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate2 sh /tmp/gciampag-sshx.sh ipengine
> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate2 sh /tmp/gciampag-sshx.sh ipengine
> 2010-09-16 01:40:02+0200 [-] about to start engines...
> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate1 sh /tmp/gciampag-sshx.sh ipengine
> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate1 sh /tmp/gciampag-sshx.sh ipengine
> 2010-09-16 01:40:02+0200 [-] about to start engines...
> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh research.inf.unisi.ch sh /tmp/gciampag-sshx.sh ipengine
> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh research.inf.unisi.ch sh /tmp/gciampag-sshx.sh ipengine
> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh research.inf.unisi.ch sh /tmp/gciampag-sshx.sh ipengine
> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh research.inf.unisi.ch sh /tmp/gciampag-sshx.sh ipengine

research.inf.unisi.ch is the host on which I have the remote port 
forwarding. This is the furl I get on research:

> giovanni@research:~$ cat .ipython/security/ipcontroller-engine.furl
> pb://vke5vvjcg6zbpejjmt6jy4stwnz5x6cm@127.0.0.1:4000,195.176.177.14:4000/4vrc5u7o5mvmoluzqfka2shvghhunwhy
> giovanni@research:~$

195.176.177.14 is the address of mate4. Is it possible that ipengine 
tries to connect to it instead of to the loopback? But manually it 
works, so I don't really know what might be the problem:

> giovanni@research:~$ ipengine
> /home/giovanni/local/lib/python2.6/site-packages/ipython-0.10-py2.6.egg/IPython/kernel/enginefc.py:78: DeprecationWarning: Importing class Referenceable1 directly from 'foolscap' is deprecated since Foolscap 0.4.3. Please import foolscap.api.Referenceable instead
>   class FCEngineReferenceFromService(Referenceable, object):
> /home/giovanni/local/lib/python2.6/site-packages/ipython-0.10-py2.6.egg/IPython/kernel/enginefc.py:511: DeprecationWarning: Importing class Referenceable1 directly from 'foolscap' is deprecated since Foolscap 0.4.3. Please import foolscap.api.Referenceable instead
>   class FCRemoteEngineRefFromService(Referenceable):
> 2010-09-16 01:46:28+0200 [-] Log opened.
> 2010-09-16 01:46:28+0200 [-] /home/giovanni/local/lib/python2.6/site-packages/ipython-0.10-py2.6.egg/IPython/kernel/scripts/ipengine.py:112: exceptions.DeprecationWarning: Importing class Tub directly from 'foolscap' is deprecated since Foolscap 0.4.3. Please import foolscap.api.Tub instead
> 2010-09-16 01:46:28+0200 [-] Using furl file: /home/giovanni/.ipython/security/ipcontroller-engine.furl
> 2010-09-16 01:46:29+0200 [Negotiation,client] engine registration succeeded, got id: 12

Best,

-- 
Giovanni L. Ciampaglia
PhD Student
University of Lugano, MACS Lab


More information about the IPython-User mailing list