[IPython-User] ipcluster in ssh mode in ipython 0.13

MinRK benjaminrk@gmail....
Wed Jul 18 11:43:03 CDT 2012


Holy crap, this is a one-character typo preventing the default value for
remote_profile_dir from being derived from the given profile_dir, leaving
it as an empty string.  Sorry about that!

You can workaround this by setting the remote profile dir explicitly with:

c.SSHEngineSetLauncher.remote_profile_dir='.ipython/profile_ssh'

(or similar)

PR #2163 <https://github.com/ipython/ipython/pull/2163> (one-character
patch) should fix it.  Can you confirm that?

-MinRK

On Wed, Jul 18, 2012 at 9:47 AM, M. Wimmer <wimmer.mike@gmail.com> wrote:
> Dear all,
>
> Having freshly installed ipython 0.13 I tried to get an ipython parallel
cluster
> running via ssh. For this I followed precisely the documentation and did
>
>  ipython profile create --parallel --profile=ssh
>
> and modified, according to the docs, in ipcluster_config.py the lines
>
>  c.IPClusterEngines.engine_launcher_class = 'SSHEngineSetLauncher'
>
>  c.SSHEngineSetLauncher.engines = {'xxx049': 1}
>
> Trying to start a cluster then misarebly fails
>
>  ipcluster start --profile=ssh --debug
>
> and I've added the debug output below. The failed `scp`-commands I can
avoid by
> writing in ipcluster_config.py
>
>  c.SSHLauncher.to_send = []
>  c.SSHLauncher.to_fetch = []
>
> As you can see from the logfile below, the engines are not started
because of
> arguments (""--profile ...") being added to the python command. I can
change the
> engine_cmd to `ipengine` but then I ran into other troubles, so I wanted
to
> discuss this step by step.
>
> What am I doing wrong? What should be added to the docs?
>
> - Michael
>
> Logfile:
> ---------------------------------------------------------
> 2012-07-18 16:40:33,001.001 [IPClusterStart] Config changed:
> 2012-07-18 16:40:33,001.001 [IPClusterStart] {'Application':
{'log_level': 10},
>  'BaseIPythonApplication': {'profile': u'ssh'}}
> 2012-07-18 16:40:33,003.003 [IPClusterStart] Using existing profile dir:
>  u'/home/wimmer/.ipython/profile_ssh'
> 2012-07-18 16:40:33,003.003 [IPClusterStart] Searching path
[u'/home/wimmer',
>  u'/home/wimmer/.ipython/profile_ssh'] for config files
> 2012-07-18 16:40:33,003.003 [IPClusterStart] Attempting to load config
file:
>  ipython_config.py
> 2012-07-18 16:40:33,005.005 [IPClusterStart] Loaded config file:
>  /home/wimmer/.ipython/profile_ssh/ipython_config.py
> 2012-07-18 16:40:33,005.005 [IPClusterStart] Attempting to load config
file:
>  ipcluster_config.py
> 2012-07-18 16:40:33,007.007 [IPClusterStart] Loaded config file:
>  /home/wimmer/.ipython/profile_ssh/ipcluster_config.py
> 2012-07-18 16:40:33,009.009 [IPClusterStart] Config changed:
> 2012-07-18 16:40:33,009.009 [IPClusterStart] {'Application':
{'log_level': 10},
>  'BaseIPythonApplication': {'profile': u'ssh'}, 'IPClusterEngines':
>  {'engine_launcher_class': 'SSHEngineSetLauncher'},
'SSHEngineSetLauncher':
>  {'engines': {'xxx049': 2}}, 'ProfileDir': {}}
> 2012-07-18 16:40:33.012 [IPClusterStart] Starting ipcluster with
[daemon=False]
> 2012-07-18 16:40:33.012 [IPClusterStart] Creating pid file:
>  /home/wimmer/.ipython/profile_ssh/pid/ipcluster.pid
> 2012-07-18 16:40:33.013 [IPClusterStart] Starting Controller with
>  LocalControllerLauncher
> 2012-07-18 16:40:33.013 [IPClusterStart] Starting LocalControllerLauncher:
>  ['/usr/bin/python', '-c', 'from IPython.parallel.apps.ipcontrollerapp
import
>  launch_new_instance; launch_new_instance()', '--profile-dir',
>  u'/home/wimmer/.ipython/profile_ssh', '--cluster-id', u'',
'--log-to-file',
>  '--log-level=20']
> 2012-07-18 16:40:33.020 [IPClusterStart] Process '/usr/bin/python'
started:
>  16130
> 2012-07-18 16:40:33.747 [IPClusterStart] 2012-07-18 16:40:33,746.746
>  [IPControllerApp] Using existing profile dir: u'/home/wimmer/.ipython
>  /profile_ssh'
> 2012-07-18 16:40:34.012 [IPClusterStart] Starting 2 Engines with
>  SSHEngineSetLauncher
> 2012-07-18 16:40:34.017 [IPClusterStart] waiting for
>  /home/wimmer/.ipython/profile_ssh/security/ipcontroller-client.json
> 2012-07-18 16:40:35.021 [IPClusterStart] sending
>  /home/wimmer/.ipython/profile_ssh/security/ipcontroller-client.json to
>  xxx049:security/ipcontroller-client.json
>  scp: security/ipcontroller-client.json: No such file or directory
> 2012-07-18 16:40:36.792 [IPClusterStart] sending
>  /home/wimmer/.ipython/profile_ssh/security/ipcontroller-engine.json to
>  xxx049:security/ipcontroller-engine.json
>  scp: security/ipcontroller-engine.json: No such file or directory
> 2012-07-18 16:40:37.957 [IPClusterStart] Starting SSHEngineLauncher:
['ssh',
>  '-tt', u'xxx049', '/usr/bin/python', '-c', 'from
>  IPython.parallel.apps.ipengineapp import launch_new_instance;
>  launch_new_instance()', '--profile-dir', u'', '--log-to-file',
>  '--log-level=20']
> 2012-07-18 16:40:37.969 [IPClusterStart] Process 'ssh' started: 16235
> 2012-07-18 16:40:38.075 [IPClusterStart] Starting SSHEngineLauncher:
['ssh',
>  '-tt', u'xxx049', '/usr/bin/python', '-c', 'from
>  IPython.parallel.apps.ipengineapp import launch_new_instance;
>  launch_new_instance()', '--profile-dir', u'', '--log-to-file',
>  '--log-level=20']
> 2012-07-18 16:40:38.085 [IPClusterStart] Process 'ssh' started: 16237
> 2012-07-18 16:40:38.086 [IPClusterStart] Process 'engine set' started:
[None,
>  None]
> 2012-07-18 16:40:38.086 [IPClusterStart] 2012-07-18 16:40:34.055
[scheduler]
>  Scheduler started [leastload]
> 2012-07-18 16:40:39.249 [IPClusterStart] bash: -c: line 0: syntax error
near
>  unexpected token `--profile-dir'
> 2012-07-18 16:40:39.249 [IPClusterStart] Connection to xxx049 closed.
> 2012-07-18 16:40:39.250 [IPClusterStart] bash: -c: line 0:
`/usr/bin/python -c
>  from IPython.parallel.apps.ipengineapp import launch_new_instance;
>  launch_new_instance() --profile-dir  --log-to-file --log-level=20'
> 2012-07-18 16:40:39.273 [IPClusterStart] Process 'ssh' stopped: {'pid':
16235,
>  'exit_code': 1}
> 2012-07-18 16:40:39.451 [IPClusterStart] bash: -c: line 0: syntax error
near
>  unexpected token `--profile-dir'
> 2012-07-18 16:40:39.451 [IPClusterStart] Connection to xxx049 closed.
> 2012-07-18 16:40:39.452 [IPClusterStart] bash: -c: line 0:
`/usr/bin/python -c
>  from IPython.parallel.apps.ipengineapp import launch_new_instance;
>  launch_new_instance() --profile-dir  --log-to-file --log-level=20'
> 2012-07-18 16:40:39.455 [IPClusterStart] Process 'ssh' stopped: {'pid':
16237,
>  'exit_code': 1}
> 2012-07-18 16:40:39.455 [IPClusterStart] Process 'engine set' stopped:
>  {'xxx049/0': {'pid': 16235, 'exit_code': 1}, 'xxx049/1': {'pid': 16237,
>  'exit_code': 1}}
> 2012-07-18 16:40:39.455 [IPClusterStart]
>             Engines shutdown early, they probably failed to connect.
>
>             Check the engine log files for output.
>
>             If your controller and engines are not on the same machine,
you probably
>             have to instruct the controller to listen on an interface
other than
> localhost.
>
>             You can set this by adding "--ip='*'" to your
> ControllerLauncher.controller_args.
>
>             Be sure to read our security docs before instructing your
controller
> to listen on
>             a public interface.
>
> 2012-07-18 16:40:39.456 [IPClusterStart] IPython cluster: stopping
> 2012-07-18 16:40:39.553 [IPClusterStart] Process '/usr/bin/python'
stopped:
>  {'pid': 16130, 'exit_code': 0}
> 2012-07-18 16:40:42.458 [IPClusterStart] Removing pid file:
>  /home/wimmer/.ipython/profile_ssh/pid/ipcluster.pid
>
>
> _______________________________________________
> IPython-User mailing list
> IPython-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-user
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20120718/52781fdd/attachment-0002.html 


More information about the IPython-User mailing list