Hi,<div><br></div><div>I am really struggling to get the parallel functionality to work on ipython 0.13. The different systems have a shared home directory, so things should be easy. I have also applied #2163. I am using EPD, so there are path issues with the </div>
<div><br></div><div>1. Create the profile</div><div><br></div><div><div>$ ipython profile create --parallel --profile=ssh </div><div>[ProfileCreate] Generating default config file: u'/home/x/.config/ipython/profile_ssh/ipython_config.py'</div>
<div>[ProfileCreate] Generating default config file: u'/home/x/.config/ipython/profile_ssh/ipython_qtconsole_config.py'</div><div>[ProfileCreate] Generating default config file: u'/home/x/.config/ipython/profile_ssh/ipython_notebook_config.py'</div>
<div>[ProfileCreate] Generating default config file: u'/home/x/.config/ipython/profile_ssh/ipcontroller_config.py'</div><div>[ProfileCreate] Generating default config file: u'/home/x/.config/ipython/profile_ssh/ipengine_config.py'</div>
<div>[ProfileCreate] Generating default config file: u'/home/x/.config/ipython/profile_ssh/ipcluster_config.py'</div><div>[ProfileCreate] Generating default config file: u'/home/x/.config/ipython/profile_ssh/iplogger_config.py'</div>
</div><div><br></div><div>2. Modify ipcluster_config.py to contain </div><div><div><br></div><div> c.IPClusterEngines.engine_launcher_class = 'SSHEngineSetLauncher' </div><div> c.SSHEngineSetLauncher.engines = {'bergen': 1} </div>
</div><div><div> c.SSHLauncher.to_send = [] </div><div> c.SSHLauncher.to_fetch = [] </div></div><div><div> c.SSHEngineSetLauncher.engine_cmd = ['/opt/epd-7.1-2-rh5-x86_64/bin/python', '-c', '\"from IPython.parallel.apps.ipengineapp import launch_new_instance; launch_new_instance()\"'] </div>
</div> c.SSHEngineSetLauncher.remote_profile_dir='/home/x/.ipython/profile_ssh'<div><span style="color:rgb(69,77,75);font-family:'Open Sans';font-size:13px;background-color:rgb(250,249,245)"><br></span></div>
<div>(I have followed the instructions here <a href="http://python.6.n6.nabble.com/IPython-User-ipcluster-in-ssh-mode-in-ipython-0-13-td4982493.html">http://python.6.n6.nabble.com/IPython-User-ipcluster-in-ssh-mode-in-ipython-0-13-td4982493.html</a>)</div>
<div><br></div><div>3. ipcluster start --profile=ssh --debug fails (see log at the bottom)</div><div><br></div><div>The log suggestst that the problem is the missing profile-dir in the ssh command:</div><div><br></div><div>
Starting SSHEngineLauncher: ['ssh', '-tt', u'bergen', '/opt/epd-7.1-2-rh5-x86_64/bin/python ', '-c ', '"from IPython.parallel.apps.ipengineapp import launch_new_instance; launch_new_instance()"', '--profile-dir', u'', '--log-to-file', '--log-level=20']</div>
<div><br></div><div><br></div><div>Everything just falls apart after that. I don't really know how to go on about this, so any help would be very helpful!</div><div><br></div><div>Thanks!</div><div>Jose</div><div><br>
</div><div><br></div><div><br></div><div>################################################################################################################################################################</div><div><br></div>
<div># stdout from ipcluster start --profile=ssh --debug </div><div><br></div><div><div>2012-08-10 19:33:42,709.709 [IPClusterStart] Config changed:</div><div>2012-08-10 19:33:42,709.709 [IPClusterStart] {'Application': {'log_level': 10}, 'BaseIPythonApplication': {'profile': u'ssh'}}</div>
<div>2012-08-10 19:33:42,777.777 [IPClusterStart] Using existing profile dir: u'/home/x/.config/ipython/profile_ssh'</div><div>2012-08-10 19:33:42,778.778 [IPClusterStart] Searching path [u'/home/x', u'/home/x/.config/ipython/profile_ssh'] for config files</div>
<div>2012-08-10 19:33:42,778.778 [IPClusterStart] Attempting to load config file: ipython_config.py</div><div>2012-08-10 19:33:42,780.780 [IPClusterStart] Loaded config file: /home/x/.config/ipython/profile_ssh/ipython_config.py</div>
<div>2012-08-10 19:33:42,780.780 [IPClusterStart] Attempting to load config file: ipcluster_config.py</div><div>2012-08-10 19:33:42,783.783 [IPClusterStart] Loaded config file: /home/x/.config/ipython/profile_ssh/ipcluster_config.py</div>
<div>2012-08-10 19:33:42,784.784 [IPClusterStart] Config changed:</div><div>2012-08-10 19:33:42,784.784 [IPClusterStart] {'BaseIPythonApplication': {'profile': u'ssh'}, 'IPClusterEngines': {'engine_launcher_class': 'SSHEngineSetLauncher'}, 'Application': {'log_level': 10}, 'ProfileDir': {}, 'SSHLauncher': {'to_fetch': [], 'to_send': []}, 'SSHEngineSetLauncher': {'remote_profile_dir': '/home/x/.config/ipython/profile_ssh', 'engine_cmd': ['/opt/epd-7.1-2-rh5-x86_64/bin/python ', '-c ', '"from IPython.parallel.apps.ipengineapp import launch_new_instance; launch_new_instance()"'], 'engines': {'bergen': 1}}}</div>
<div>2012-08-10 19:33:42.788 [IPClusterStart] Starting ipcluster with [daemon=False]</div><div>2012-08-10 19:33:42.802 [IPClusterStart] Creating pid file: /home/x/.config/ipython/profile_ssh/pid/ipcluster.pid</div><div>2012-08-10 19:33:42.832 [IPClusterStart] Starting Controller with LocalControllerLauncher</div>
<div>2012-08-10 19:33:42.832 [IPClusterStart] Starting LocalControllerLauncher: ['/opt/epd-7.1-2-rh5-x86_64/bin/python', '-c', 'from IPython.parallel.apps.ipcontrollerapp import launch_new_instance; launch_new_instance()', '--profile-dir', u'/home/x/.config/ipython/profile_ssh', '--cluster-id', u'', '--log-to-file', '--log-level=20']</div>
<div>2012-08-10 19:33:42.841 [IPClusterStart] Process '/opt/epd-7.1-2-rh5-x86_64/bin/python' started: 16905</div><div>2012-08-10 19:33:43.546 [IPClusterStart] 2012-08-10 19:33:43,545.545 [IPControllerApp] Using existing profile dir: u'/home/x/.config/ipython/profile_ssh'</div>
<div>2012-08-10 19:33:43.789 [IPClusterStart] Starting 1 Engines with SSHEngineSetLauncher</div><div>2012-08-10 19:33:43.791 [IPClusterStart] Starting SSHEngineLauncher: ['ssh', '-tt', u'bergen', '/opt/epd-7.1-2-rh5-x86_64/bin/python ', '-c ', '"from IPython.parallel.apps.ipengineapp import launch_new_instance; launch_new_instance()"', '--profile-dir', u'', '--log-to-file', '--log-level=20']</div>
<div>2012-08-10 19:33:43.801 [IPClusterStart] Process 'ssh' started: 16920</div><div>2012-08-10 19:33:43.801 [IPClusterStart] Process 'engine set' started: [None]</div><div>2012-08-10 19:33:43.892 [IPClusterStart] tcgetattr: Invalid argument</div>
<div>2012-08-10 19:33:43.996 [IPClusterStart] 2012-08-10 19:33:43.996 [scheduler] Scheduler started [leastload]</div><div>2012-08-10 19:33:45.133 [IPClusterStart] usage: -c [-h] [--ip ENGINEFACTORY.IP] [--work-dir IPENGINEAPP.WORK_DIR]</div>
<div>2012-08-10 19:33:45.133 [IPClusterStart] [--file IPENGINEAPP.URL_FILE] [--port ENGINEFACTORY.REGPORT]</div><div>2012-08-10 19:33:45.133 [IPClusterStart] [--profile-dir PROFILEDIR.LOCATION]</div><div>
2012-08-10 19:33:45.134 [IPClusterStart] [--log-to-file [IPENGINEAPP.LOG_TO_FILE]]</div><div>2012-08-10 19:33:45.134 [IPClusterStart] [--sshkey ENGINEFACTORY.SSHKEY] [--location ENGINEFACTORY.LOCATION]</div>
<div>2012-08-10 19:33:45.134 [IPClusterStart] [--clean-logs IPENGINEAPP.CLEAN_LOGS]</div><div>2012-08-10 19:33:45.134 [IPClusterStart] [--profile IPENGINEAPP.PROFILE]</div><div>2012-08-10 19:33:45.134 [IPClusterStart] [--cluster-id IPENGINEAPP.CLUSTER_ID] [--mpi MPI.USE]</div>
<div>2012-08-10 19:33:45.134 [IPClusterStart] [--ssh ENGINEFACTORY.SSHSERVER] [-c IPENGINEAPP.STARTUP_COMMAND]</div><div>2012-08-10 19:33:45.134 [IPClusterStart] [--ident SESSION.SESSION] [--ipython-dir IPENGINEAPP.IPYTHON_DIR]</div>
<div>2012-08-10 19:33:45.134 [IPClusterStart] [--url ENGINEFACTORY.URL] [--log-level IPENGINEAPP.LOG_LEVEL]</div><div>2012-08-10 19:33:45.135 [IPClusterStart] [-s IPENGINEAPP.STARTUP_SCRIPT] [--log-url IPENGINEAPP.LOG_URL]</div>
<div>2012-08-10 19:33:45.135 [IPClusterStart] [--timeout ENGINEFACTORY.TIMEOUT] [--keyfile SESSION.KEYFILE]</div><div>2012-08-10 19:33:45.135 [IPClusterStart] [--transport ENGINEFACTORY.TRANSPORT] [--user SESSION.USERNAME]</div>
<div>2012-08-10 19:33:45.135 [IPClusterStart] [--no-secure] [--secure] [--quiet] [--init] [--debug]</div><div>2012-08-10 19:33:45.135 [IPClusterStart] -c: error: argument --profile-dir: expected one argument</div>
<div>2012-08-10 19:33:45.162 [IPClusterStart] Connection to bergen closed.</div><div>2012-08-10 19:33:45.163 [IPClusterStart] Process 'ssh' stopped: {'pid': 16920, 'exit_code': 2}</div><div>2012-08-10 19:33:45.163 [IPClusterStart] Process 'engine set' stopped: {'bergen/0': {'pid': 16920, 'exit_code': 2}}</div>
<div>2012-08-10 19:33:45.163 [IPClusterStart] </div><div> Engines shutdown early, they probably failed to connect.</div><div> </div><div> Check the engine log files for output.</div><div> </div>
<div> If your controller and engines are not on the same machine, you probably</div><div> have to instruct the controller to listen on an interface other than localhost.</div><div> </div><div>
You can set this by adding "--ip='*'" to your ControllerLauncher.controller_args.</div><div> </div><div> Be sure to read our security docs before instructing your controller to listen on</div>
<div> a public interface.</div><div> </div><div>2012-08-10 19:33:45.164 [IPClusterStart] IPython cluster: stopping</div><div>2012-08-10 19:33:45.222 [IPClusterStart] Traceback (most recent call last):</div>
<div>2012-08-10 19:33:45.222 [IPClusterStart] File "<string>", line 1, in <module></div><div>2012-08-10 19:33:45.222 [IPClusterStart] File "/home/x/.local/lib/python2.7/site-packages/IPython/parallel/apps/ipcontrollerapp.py", line 487, in launch_new_instance</div>
<div>2012-08-10 19:33:45.222 [IPClusterStart] app.start()</div><div>2012-08-10 19:33:45.222 [IPClusterStart] File "/home/x/.local/lib/python2.7/site-packages/IPython/parallel/apps/ipcontrollerapp.py", line 461, in start</div>
<div>2012-08-10 19:33:45.223 [IPClusterStart] self.factory.loop.start()</div><div>2012-08-10 19:33:45.223 [IPClusterStart] File "/opt/epd-7.1-2-rh5-x86_64/lib/python2.7/site-packages/zmq/eventloop/ioloop.py", line 251, in start</div>
<div>2012-08-10 19:33:45.223 [IPClusterStart] event_pairs = self._impl.poll(poll_timeout)</div><div>2012-08-10 19:33:45.223 [IPClusterStart] File "poll.pyx", line 189, in zmq.core.poll.Poller.poll (zmq/core/poll.c:2120)</div>
<div>2012-08-10 19:33:45.223 [IPClusterStart] File "poll.pyx", line 101, in zmq.core.poll._poll (zmq/core/poll.c:1400)</div><div>2012-08-10 19:33:45.223 [IPClusterStart] ZMQError: Interrupted system call</div>
<div>2012-08-10 19:33:45.223 [IPClusterStart] </div><div>2012-08-10 19:33:45.224 [IPClusterStart] If you suspect this is an IPython bug, please report it at:</div><div>2012-08-10 19:33:45.224 [IPClusterStart] <a href="https://github.com/ipython/ipython/issues">https://github.com/ipython/ipython/issues</a></div>
<div>2012-08-10 19:33:45.224 [IPClusterStart] or send an email to the mailing list at <a href="mailto:ipython-dev@scipy.org">ipython-dev@scipy.org</a></div><div>2012-08-10 19:33:45.224 [IPClusterStart] </div><div>2012-08-10 19:33:45.224 [IPClusterStart] You can print a more detailed traceback right now with "%tb", or use "%debug"</div>
<div>2012-08-10 19:33:45.225 [IPClusterStart] to interactively debug it.</div><div>2012-08-10 19:33:45.225 [IPClusterStart] </div><div>2012-08-10 19:33:45.225 [IPClusterStart] Extra-detailed tracebacks for bug-reporting purposes can be enabled via:</div>
<div>2012-08-10 19:33:45.225 [IPClusterStart] c.Application.verbose_crash=True</div><div>2012-08-10 19:33:45.225 [IPClusterStart] </div><div>2012-08-10 19:33:45.260 [IPClusterStart] Process '/opt/epd-7.1-2-rh5-x86_64/bin/python' stopped: {'pid': 16905, 'exit_code': 1}</div>
<div>2012-08-10 19:33:48.164 [IPClusterStart] Removing pid file: /home/x/.config/ipython/profile_ssh/pid/ipcluster.pid</div></div><div><br></div><div><br></div>