[IPython-User] ipcluster ssh and remote port forwarding

Brian Granger ellisonbg@gmail....
Thu Sep 16 09:30:52 CDT 2010


On Thu, Sep 16, 2010 at 2:05 AM, Giovanni Luca Ciampaglia
<giovanni.luca.ciampaglia@usi.ch> wrote:
>  Hi,
> after some trial and error I found out it was a problem related to a missing
> environment variable. Thanks for the help!

Glad you got it figured out!

Brian

> G
>
> Il 16.09.10 01.48, Giovanni Luca Ciampaglia ha scritto:
>>
>> On 16/09/2010 00:58, Brian Granger wrote:
>>
>>>> How do I use the same trick, but with ipcluster ssh? That is, how do I
>>>> tell to ipcluster which port the engines should connect to so that they
>>>> can talk to the ipcontroller?
>>>
>>> Assuming you are using IPython version 0.10, the ports are controlled
>>> by these flags:
>>>
>>>    --client-port CLIENT_PORT
>>>                          the port the controller will listen on for
>>> client
>>>                          connections
>>>    --engine-port ENGINE_PORT
>>>                          the port the controller will listen on for
>>> engine
>>>                          connections
>>>
>> Hi Brian,
>> Yes, I have 0.10. I tried using those options and reusing the furl files
>> (-r).
>>
>> ipcluster ssh tries to start the ipengine instances on host B but it
>> appear to not succeed in that. However, once the ipcontroller has
>> started, if I ssh host B and start manually the engines, they do
>> connect. After removing all furl files I do:
>>
>>> [gciampag@mate4 wikimodels_devel]$ ipcluster ssh --clusterfile cluster.py
>>> --engine-port=4000 --client-port=4001 -r
>>> 2010-09-16 01:40:01+0200 [-] Log opened.
>>> 2010-09-16 01:40:01+0200 [-] Process ['ipcontroller',
>>> '--logfile=/home/gciampag/.ipython/log/ipcontroller', '-r',
>>> '--client-port=4001', '--engine-port=4000'] has started with pid=3108
>>> 2010-09-16 01:40:01+0200 [-] Waiting for controller to finish starting...
>>> 2010-09-16 01:40:01+0200 [-] Controller started
>>> 2010-09-16 01:40:01+0200 [-] Copying furl file: scp
>>> ~/.ipython/security/ipcontroller-engine.furl mate1:.ipython/security/
>>> 2010-09-16 01:40:01+0200 [-] Copying furl file: scp
>>> ~/.ipython/security/ipcontroller-engine.furl mate2:.ipython/security/
>>> 2010-09-16 01:40:01+0200 [-] Copying furl file: scp
>>> ~/.ipython/security/ipcontroller-engine.furl mate3:.ipython/security/
>>> 2010-09-16 01:40:01+0200 [-] Copying furl file: scp
>>> ~/.ipython/security/ipcontroller-engine.furl mate4:.ipython/security/
>>> 2010-09-16 01:40:01+0200 [-] Copying furl file: scp
>>> ~/.ipython/security/ipcontroller-engine.furl
>>> research.inf.unisi.ch:.ipython/security/
>>> 2010-09-16 01:40:01+0200 [-]
>>> 2010-09-16 01:40:01+0200 [-] Copying sshx: scp /tmp/gciampag-main-sshx.sh
>>> mate2:/tmp/gciampag-sshx.sh
>>> 2010-09-16 01:40:01+0200 [-]
>>> 2010-09-16 01:40:01+0200 [-] Copying sshx: scp /tmp/gciampag-main-sshx.sh
>>> mate4:/tmp/gciampag-sshx.sh
>>> 2010-09-16 01:40:01+0200 [-]
>>> 2010-09-16 01:40:01+0200 [-] Copying sshx: scp /tmp/gciampag-main-sshx.sh
>>> mate1:/tmp/gciampag-sshx.sh
>>> 2010-09-16 01:40:01+0200 [-]
>>> 2010-09-16 01:40:01+0200 [-] Copying sshx: scp /tmp/gciampag-main-sshx.sh
>>> mate3:/tmp/gciampag-sshx.sh
>>> 2010-09-16 01:40:01+0200 [-]
>>> 2010-09-16 01:40:01+0200 [-] Copying sshx: scp /tmp/gciampag-main-sshx.sh
>>> research.inf.unisi.ch:/tmp/gciampag-sshx.sh
>>> 2010-09-16 01:40:02+0200 [-] about to start engines...
>>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate4 sh
>>> /tmp/gciampag-sshx.sh ipengine
>>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate4 sh
>>> /tmp/gciampag-sshx.sh ipengine
>>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate4 sh
>>> /tmp/gciampag-sshx.sh ipengine
>>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate4 sh
>>> /tmp/gciampag-sshx.sh ipengine
>>> 2010-09-16 01:40:02+0200 [-] about to start engines...
>>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate3 sh
>>> /tmp/gciampag-sshx.sh ipengine
>>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate3 sh
>>> /tmp/gciampag-sshx.sh ipengine
>>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate3 sh
>>> /tmp/gciampag-sshx.sh ipengine
>>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate3 sh
>>> /tmp/gciampag-sshx.sh ipengine
>>> 2010-09-16 01:40:02+0200 [-] about to start engines...
>>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate2 sh
>>> /tmp/gciampag-sshx.sh ipengine
>>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate2 sh
>>> /tmp/gciampag-sshx.sh ipengine
>>> 2010-09-16 01:40:02+0200 [-] about to start engines...
>>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate1 sh
>>> /tmp/gciampag-sshx.sh ipengine
>>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh mate1 sh
>>> /tmp/gciampag-sshx.sh ipengine
>>> 2010-09-16 01:40:02+0200 [-] about to start engines...
>>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh research.inf.unisi.ch
>>> sh /tmp/gciampag-sshx.sh ipengine
>>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh research.inf.unisi.ch
>>> sh /tmp/gciampag-sshx.sh ipengine
>>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh research.inf.unisi.ch
>>> sh /tmp/gciampag-sshx.sh ipengine
>>> 2010-09-16 01:40:02+0200 [-] Starting engines: ssh research.inf.unisi.ch
>>> sh /tmp/gciampag-sshx.sh ipengine
>>
>> research.inf.unisi.ch is the host on which I have the remote port
>> forwarding. This is the furl I get on research:
>>
>>> giovanni@research:~$ cat .ipython/security/ipcontroller-engine.furl
>>>
>>> pb://vke5vvjcg6zbpejjmt6jy4stwnz5x6cm@127.0.0.1:4000,195.176.177.14:4000/4vrc5u7o5mvmoluzqfka2shvghhunwhy
>>> giovanni@research:~$
>>
>> 195.176.177.14 is the address of mate4. Is it possible that ipengine
>> tries to connect to it instead of to the loopback? But manually it
>> works, so I don't really know what might be the problem:
>>
>>> giovanni@research:~$ ipengine
>>>
>>> /home/giovanni/local/lib/python2.6/site-packages/ipython-0.10-py2.6.egg/IPython/kernel/enginefc.py:78:
>>> DeprecationWarning: Importing class Referenceable1 directly from 'foolscap'
>>> is deprecated since Foolscap 0.4.3. Please import foolscap.api.Referenceable
>>> instead
>>>   class FCEngineReferenceFromService(Referenceable, object):
>>>
>>> /home/giovanni/local/lib/python2.6/site-packages/ipython-0.10-py2.6.egg/IPython/kernel/enginefc.py:511:
>>> DeprecationWarning: Importing class Referenceable1 directly from 'foolscap'
>>> is deprecated since Foolscap 0.4.3. Please import foolscap.api.Referenceable
>>> instead
>>>   class FCRemoteEngineRefFromService(Referenceable):
>>> 2010-09-16 01:46:28+0200 [-] Log opened.
>>> 2010-09-16 01:46:28+0200 [-]
>>> /home/giovanni/local/lib/python2.6/site-packages/ipython-0.10-py2.6.egg/IPython/kernel/scripts/ipengine.py:112:
>>> exceptions.DeprecationWarning: Importing class Tub directly from 'foolscap'
>>> is deprecated since Foolscap 0.4.3. Please import foolscap.api.Tub instead
>>> 2010-09-16 01:46:28+0200 [-] Using furl file:
>>> /home/giovanni/.ipython/security/ipcontroller-engine.furl
>>> 2010-09-16 01:46:29+0200 [Negotiation,client] engine registration
>>> succeeded, got id: 12
>>
>> Best,
>>
>
>
> --
> Giovanni L. Ciampaglia
> PhD Student
> University of Lugano, MACS Lab
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu
ellisonbg@gmail.com


More information about the IPython-User mailing list