[IPython-User] ipcluster ssh and remote port forwarding

Giovanni Luca Ciampaglia giovanni.luca.ciampaglia@usi...
Thu Sep 16 04:05:11 CDT 2010


  Hi,
after some trial and error I found out it was a problem related to a 
missing environment variable. Thanks for the help!

G

Il 16.09.10 01.48, Giovanni Luca Ciampaglia ha scritto:
> On 16/09/2010 00:58, Brian Granger wrote:
>
>>> How do I use the same trick, but with ipcluster ssh? That is, how do I
>>> tell to ipcluster which port the engines should connect to so that they
>>> can talk to the ipcontroller?
>> Assuming you are using IPython version 0.10, the ports are controlled
>> by these flags:
>>
>>     --client-port CLIENT_PORT
>>                           the port the controller will listen on for client
>>                           connections
>>     --engine-port ENGINE_PORT
>>                           the port the controller will listen on for engine
>>                           connections
>>
> Hi Brian,
> Yes, I have 0.10. I tried using those options and reusing the furl files
> (-r).
>
> ipcluster ssh tries to start the ipengine instances on host B but it
> appear to not succeed in that. However, once the ipcontroller has
> started, if I ssh host B and start manually the engines, they do
> connect. After removing all furl files I do:
>
>> [gciampag@mate4 wikimodels_devel]$ ipcluster ssh --clusterfile cluster.py --engine-port=4000 --client-port=4001 -r
>> 2010-09-16 01:40:01+0200 [-] Log opened.
>> 2010-09-16 01:40:01+0200 [-] Process ['ipcontroller', '--logfile=/home/gciampag/.ipython/log/ipcontroller', '-r', '--client-port=4001', '--engine-port=4000'] has started with pid=3108
>> 2010-09-16 01:40:01+0200 [-] Waiting for controller to finish starting...
>> 2010-09-16 01:40:01+0200 [-] Controller started
>> 2010-09-16 01:40:01+0200 [-] Copying furl file: scp ~/.ipython/security/ipcontroller-engine.furl mate1:.ipython/security/
>> 2010-09-16 01:40:01+0200 [-] Copying furl file: scp ~/.ipython/security/ipcontroller-engine.furl mate2:.ipython/security/
>> 2010-09-16 01:40:01+0200 [-] Copying furl file: scp ~/.ipython/security/ipcontroller-engine.furl mate3:.ipython/security/
>> 2010-09-16 01:40:01+0200 [-] Copying furl file: scp ~/.ipython/security/ipcontroller-engine.furl mate4:.ipython/security/
>> 2010-09-16 01:40:01+0200 [-] Copying furl file: scp ~/.ipython/security/ipcontroller-engine.furl research.inf.unisi.ch:.ipython/security/
>> 2010-09-16 01:40:01+0200 [-]
>> 2010-09-16 01:40:01+0200 [-] Copying sshx: scp /tmp/gciampag-main-sshx.sh mate2:/tmp/gciampag-sshx.sh
>> 2010-09-16 01:40:01+0200 [-]
>> 2010-09-16 01:40:01+0200 [-] Copying sshx: scp /tmp/gciampag-main-sshx.sh mate4:/tmp/gciampag-sshx.sh
>> 2010-09-16 01:40:01+0200 [-]
>> 2010-09-16 01:40:01+0200 [-] Copying sshx: scp /tmp/gciampag-main-sshx.sh mate1:/tmp/gciampag-sshx.sh
>> 2010-09-16 01:40:01+0200 [-]
>> 2010-09-16 01:40:01+0200 [-] Copying sshx: scp /tmp/gciampag-main-sshx.sh mate3:/tmp/gciampag-sshx.sh
>> 2010-09-16 01:40:01+0200 [-]
>> 2010-09-16 01:40:01+0200 [-] Copying sshx: scp /tmp/gciampag-main-sshx.sh research.inf.unisi.ch:/tmp/gciampag-sshx.sh
>> 2010-09-16 01:40:02+0200 [-] about to start engines...
>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate4 sh /tmp/gciampag-sshx.sh ipengine
>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate4 sh /tmp/gciampag-sshx.sh ipengine
>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate4 sh /tmp/gciampag-sshx.sh ipengine
>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate4 sh /tmp/gciampag-sshx.sh ipengine
>> 2010-09-16 01:40:02+0200 [-] about to start engines...
>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate3 sh /tmp/gciampag-sshx.sh ipengine
>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate3 sh /tmp/gciampag-sshx.sh ipengine
>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate3 sh /tmp/gciampag-sshx.sh ipengine
>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate3 sh /tmp/gciampag-sshx.sh ipengine
>> 2010-09-16 01:40:02+0200 [-] about to start engines...
>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate2 sh /tmp/gciampag-sshx.sh ipengine
>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate2 sh /tmp/gciampag-sshx.sh ipengine
>> 2010-09-16 01:40:02+0200 [-] about to start engines...
>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate1 sh /tmp/gciampag-sshx.sh ipengine
>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate1 sh /tmp/gciampag-sshx.sh ipengine
>> 2010-09-16 01:40:02+0200 [-] about to start engines...
>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh research.inf.unisi.ch sh /tmp/gciampag-sshx.sh ipengine
>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh research.inf.unisi.ch sh /tmp/gciampag-sshx.sh ipengine
>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh research.inf.unisi.ch sh /tmp/gciampag-sshx.sh ipengine
>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh research.inf.unisi.ch sh /tmp/gciampag-sshx.sh ipengine
> research.inf.unisi.ch is the host on which I have the remote port
> forwarding. This is the furl I get on research:
>
>> giovanni@research:~$ cat .ipython/security/ipcontroller-engine.furl
>> pb://vke5vvjcg6zbpejjmt6jy4stwnz5x6cm@127.0.0.1:4000,195.176.177.14:4000/4vrc5u7o5mvmoluzqfka2shvghhunwhy
>> giovanni@research:~$
> 195.176.177.14 is the address of mate4. Is it possible that ipengine
> tries to connect to it instead of to the loopback? But manually it
> works, so I don't really know what might be the problem:
>
>> giovanni@research:~$ ipengine
>> /home/giovanni/local/lib/python2.6/site-packages/ipython-0.10-py2.6.egg/IPython/kernel/enginefc.py:78: DeprecationWarning: Importing class Referenceable1 directly from 'foolscap' is deprecated since Foolscap 0.4.3. Please import foolscap.api.Referenceable instead
>>    class FCEngineReferenceFromService(Referenceable, object):
>> /home/giovanni/local/lib/python2.6/site-packages/ipython-0.10-py2.6.egg/IPython/kernel/enginefc.py:511: DeprecationWarning: Importing class Referenceable1 directly from 'foolscap' is deprecated since Foolscap 0.4.3. Please import foolscap.api.Referenceable instead
>>    class FCRemoteEngineRefFromService(Referenceable):
>> 2010-09-16 01:46:28+0200 [-] Log opened.
>> 2010-09-16 01:46:28+0200 [-] /home/giovanni/local/lib/python2.6/site-packages/ipython-0.10-py2.6.egg/IPython/kernel/scripts/ipengine.py:112: exceptions.DeprecationWarning: Importing class Tub directly from 'foolscap' is deprecated since Foolscap 0.4.3. Please import foolscap.api.Tub instead
>> 2010-09-16 01:46:28+0200 [-] Using furl file: /home/giovanni/.ipython/security/ipcontroller-engine.furl
>> 2010-09-16 01:46:29+0200 [Negotiation,client] engine registration succeeded, got id: 12
> Best,
>


-- 
Giovanni L. Ciampaglia
PhD Student
University of Lugano, MACS Lab



More information about the IPython-User mailing list