Holy crap, this is a one-character typo preventing the default value for remote_profile_dir from being derived from the given profile_dir, leaving it as an empty string. Sorry about that!<br><br>You can workaround this by setting the remote profile dir explicitly with:<br>
<br>c.SSHEngineSetLauncher.remote_profile_dir='.ipython/profile_ssh'<div><br></div><div>(or similar)<br><br><a href="https://github.com/ipython/ipython/pull/2163">PR #2163</a> (one-character patch) should fix it. Can you confirm that?<br>
<br>-MinRK<br><br>On Wed, Jul 18, 2012 at 9:47 AM, M. Wimmer <<a href="mailto:wimmer.mike@gmail.com">wimmer.mike@gmail.com</a>> wrote:<br>> Dear all,<br>><br>> Having freshly installed ipython 0.13 I tried to get an ipython parallel cluster<br>
> running via ssh. For this I followed precisely the documentation and did<br>><br>> ipython profile create --parallel --profile=ssh<br>><br>> and modified, according to the docs, in ipcluster_config.py the lines<br>
><br>> c.IPClusterEngines.engine_launcher_class = 'SSHEngineSetLauncher'<br>><br>> c.SSHEngineSetLauncher.engines = {'xxx049': 1}<br>><br>> Trying to start a cluster then misarebly fails<br>
><br>> ipcluster start --profile=ssh --debug<br>><br>> and I've added the debug output below. The failed `scp`-commands I can avoid by<br>> writing in ipcluster_config.py<br>><br>> c.SSHLauncher.to_send = []<br>
> c.SSHLauncher.to_fetch = []<br>><br>> As you can see from the logfile below, the engines are not started because of<br>> arguments (""--profile ...") being added to the python command. I can change the<br>
> engine_cmd to `ipengine` but then I ran into other troubles, so I wanted to<br>> discuss this step by step.<br>><br>> What am I doing wrong? What should be added to the docs?<br>><br>> - Michael<br>><br>
> Logfile:<br>> ---------------------------------------------------------<br>> 2012-07-18 16:40:33,001.001 [IPClusterStart] Config changed:<br>> 2012-07-18 16:40:33,001.001 [IPClusterStart] {'Application': {'log_level': 10},<br>
> 'BaseIPythonApplication': {'profile': u'ssh'}}<br>> 2012-07-18 16:40:33,003.003 [IPClusterStart] Using existing profile dir:<br>> u'/home/wimmer/.ipython/profile_ssh'<br>> 2012-07-18 16:40:33,003.003 [IPClusterStart] Searching path [u'/home/wimmer',<br>
> u'/home/wimmer/.ipython/profile_ssh'] for config files<br>> 2012-07-18 16:40:33,003.003 [IPClusterStart] Attempting to load config file:<br>> ipython_config.py<br>> 2012-07-18 16:40:33,005.005 [IPClusterStart] Loaded config file:<br>
> /home/wimmer/.ipython/profile_ssh/ipython_config.py<br>> 2012-07-18 16:40:33,005.005 [IPClusterStart] Attempting to load config file:<br>> ipcluster_config.py<br>> 2012-07-18 16:40:33,007.007 [IPClusterStart] Loaded config file:<br>
> /home/wimmer/.ipython/profile_ssh/ipcluster_config.py<br>> 2012-07-18 16:40:33,009.009 [IPClusterStart] Config changed:<br>> 2012-07-18 16:40:33,009.009 [IPClusterStart] {'Application': {'log_level': 10},<br>
> 'BaseIPythonApplication': {'profile': u'ssh'}, 'IPClusterEngines':<br>> {'engine_launcher_class': 'SSHEngineSetLauncher'}, 'SSHEngineSetLauncher':<br>> {'engines': {'xxx049': 2}}, 'ProfileDir': {}}<br>
> 2012-07-18 16:40:33.012 [IPClusterStart] Starting ipcluster with [daemon=False]<br>> 2012-07-18 16:40:33.012 [IPClusterStart] Creating pid file:<br>> /home/wimmer/.ipython/profile_ssh/pid/ipcluster.pid<br>> 2012-07-18 16:40:33.013 [IPClusterStart] Starting Controller with<br>
> LocalControllerLauncher<br>> 2012-07-18 16:40:33.013 [IPClusterStart] Starting LocalControllerLauncher:<br>> ['/usr/bin/python', '-c', 'from IPython.parallel.apps.ipcontrollerapp import<br>
> launch_new_instance; launch_new_instance()', '--profile-dir',<br>> u'/home/wimmer/.ipython/profile_ssh', '--cluster-id', u'', '--log-to-file',<br>> '--log-level=20']<br>
> 2012-07-18 16:40:33.020 [IPClusterStart] Process '/usr/bin/python' started:<br>> 16130<br>> 2012-07-18 16:40:33.747 [IPClusterStart] 2012-07-18 16:40:33,746.746<br>> [IPControllerApp] Using existing profile dir: u'/home/wimmer/.ipython<br>
> /profile_ssh'<br>> 2012-07-18 16:40:34.012 [IPClusterStart] Starting 2 Engines with<br>> SSHEngineSetLauncher<br>> 2012-07-18 16:40:34.017 [IPClusterStart] waiting for<br>> /home/wimmer/.ipython/profile_ssh/security/ipcontroller-client.json<br>
> 2012-07-18 16:40:35.021 [IPClusterStart] sending<br>> /home/wimmer/.ipython/profile_ssh/security/ipcontroller-client.json to<br>> xxx049:security/ipcontroller-client.json<br>> scp: security/ipcontroller-client.json: No such file or directory<br>
> 2012-07-18 16:40:36.792 [IPClusterStart] sending<br>> /home/wimmer/.ipython/profile_ssh/security/ipcontroller-engine.json to<br>> xxx049:security/ipcontroller-engine.json<br>> scp: security/ipcontroller-engine.json: No such file or directory<br>
> 2012-07-18 16:40:37.957 [IPClusterStart] Starting SSHEngineLauncher: ['ssh',<br>> '-tt', u'xxx049', '/usr/bin/python', '-c', 'from<br>> IPython.parallel.apps.ipengineapp import launch_new_instance;<br>
> launch_new_instance()', '--profile-dir', u'', '--log-to-file',<br>> '--log-level=20']<br>> 2012-07-18 16:40:37.969 [IPClusterStart] Process 'ssh' started: 16235<br>
> 2012-07-18 16:40:38.075 [IPClusterStart] Starting SSHEngineLauncher: ['ssh',<br>> '-tt', u'xxx049', '/usr/bin/python', '-c', 'from<br>> IPython.parallel.apps.ipengineapp import launch_new_instance;<br>
> launch_new_instance()', '--profile-dir', u'', '--log-to-file',<br>> '--log-level=20']<br>> 2012-07-18 16:40:38.085 [IPClusterStart] Process 'ssh' started: 16237<br>
> 2012-07-18 16:40:38.086 [IPClusterStart] Process 'engine set' started: [None,<br>> None]<br>> 2012-07-18 16:40:38.086 [IPClusterStart] 2012-07-18 16:40:34.055 [scheduler]<br>> Scheduler started [leastload]<br>
> 2012-07-18 16:40:39.249 [IPClusterStart] bash: -c: line 0: syntax error near<br>> unexpected token `--profile-dir'<br>> 2012-07-18 16:40:39.249 [IPClusterStart] Connection to xxx049 closed.<br>> 2012-07-18 16:40:39.250 [IPClusterStart] bash: -c: line 0: `/usr/bin/python -c<br>
> from IPython.parallel.apps.ipengineapp import launch_new_instance;<br>> launch_new_instance() --profile-dir --log-to-file --log-level=20'<br>> 2012-07-18 16:40:39.273 [IPClusterStart] Process 'ssh' stopped: {'pid': 16235,<br>
> 'exit_code': 1}<br>> 2012-07-18 16:40:39.451 [IPClusterStart] bash: -c: line 0: syntax error near<br>> unexpected token `--profile-dir'<br>> 2012-07-18 16:40:39.451 [IPClusterStart] Connection to xxx049 closed.<br>
> 2012-07-18 16:40:39.452 [IPClusterStart] bash: -c: line 0: `/usr/bin/python -c<br>> from IPython.parallel.apps.ipengineapp import launch_new_instance;<br>> launch_new_instance() --profile-dir --log-to-file --log-level=20'<br>
> 2012-07-18 16:40:39.455 [IPClusterStart] Process 'ssh' stopped: {'pid': 16237,<br>> 'exit_code': 1}<br>> 2012-07-18 16:40:39.455 [IPClusterStart] Process 'engine set' stopped:<br>
> {'xxx049/0': {'pid': 16235, 'exit_code': 1}, 'xxx049/1': {'pid': 16237,<br>> 'exit_code': 1}}<br>> 2012-07-18 16:40:39.455 [IPClusterStart]<br>> Engines shutdown early, they probably failed to connect.<br>
><br>> Check the engine log files for output.<br>><br>> If your controller and engines are not on the same machine, you probably<br>> have to instruct the controller to listen on an interface other than<br>
> localhost.<br>><br>> You can set this by adding "--ip='*'" to your<br>> ControllerLauncher.controller_args.<br>><br>> Be sure to read our security docs before instructing your controller<br>
> to listen on<br>> a public interface.<br>><br>> 2012-07-18 16:40:39.456 [IPClusterStart] IPython cluster: stopping<br>> 2012-07-18 16:40:39.553 [IPClusterStart] Process '/usr/bin/python' stopped:<br>
> {'pid': 16130, 'exit_code': 0}<br>> 2012-07-18 16:40:42.458 [IPClusterStart] Removing pid file:<br>> /home/wimmer/.ipython/profile_ssh/pid/ipcluster.pid<br>><br>><br>> _______________________________________________<br>
> IPython-User mailing list<br>> <a href="mailto:IPython-User@scipy.org">IPython-User@scipy.org</a><br>> <a href="http://mail.scipy.org/mailman/listinfo/ipython-user">http://mail.scipy.org/mailman/listinfo/ipython-user</a><br>
<br></div>