Hi,<div><br></div><div>I am really struggling to get the parallel functionality to work on ipython 0.13. The different systems have a shared home directory, so things should be easy. I have also applied #2163. I am using EPD, so there are path issues with the </div>
<div><br></div><div>1. Create the profile</div><div><br></div><div><div>$ ipython profile create --parallel --profile=ssh </div><div>[ProfileCreate] Generating default config file: u&#39;/home/x/.config/ipython/profile_ssh/ipython_config.py&#39;</div>
<div>[ProfileCreate] Generating default config file: u&#39;/home/x/.config/ipython/profile_ssh/ipython_qtconsole_config.py&#39;</div><div>[ProfileCreate] Generating default config file: u&#39;/home/x/.config/ipython/profile_ssh/ipython_notebook_config.py&#39;</div>
<div>[ProfileCreate] Generating default config file: u&#39;/home/x/.config/ipython/profile_ssh/ipcontroller_config.py&#39;</div><div>[ProfileCreate] Generating default config file: u&#39;/home/x/.config/ipython/profile_ssh/ipengine_config.py&#39;</div>
<div>[ProfileCreate] Generating default config file: u&#39;/home/x/.config/ipython/profile_ssh/ipcluster_config.py&#39;</div><div>[ProfileCreate] Generating default config file: u&#39;/home/x/.config/ipython/profile_ssh/iplogger_config.py&#39;</div>
</div><div><br></div><div>2. Modify ipcluster_config.py to contain </div><div><div><br></div><div> c.IPClusterEngines.engine_launcher_class = &#39;SSHEngineSetLauncher&#39; </div><div> c.SSHEngineSetLauncher.engines = {&#39;bergen&#39;: 1} </div>
</div><div><div> c.SSHLauncher.to_send = [] </div><div> c.SSHLauncher.to_fetch = [] </div></div><div><div> c.SSHEngineSetLauncher.engine_cmd = [&#39;/opt/epd-7.1-2-rh5-x86_64/bin/python&#39;, &#39;-c&#39;, &#39;\&quot;from IPython.parallel.apps.ipengineapp import launch_new_instance;   launch_new_instance()\&quot;&#39;] </div>
</div> c.SSHEngineSetLauncher.remote_profile_dir=&#39;/home/x/.ipython/profile_ssh&#39;<div><span style="color:rgb(69,77,75);font-family:&#39;Open Sans&#39;;font-size:13px;background-color:rgb(250,249,245)"><br></span></div>
<div>(I have followed the instructions here <a href="http://python.6.n6.nabble.com/IPython-User-ipcluster-in-ssh-mode-in-ipython-0-13-td4982493.html">http://python.6.n6.nabble.com/IPython-User-ipcluster-in-ssh-mode-in-ipython-0-13-td4982493.html</a>)</div>
<div><br></div><div>3.  ipcluster start --profile=ssh --debug fails (see log at the bottom)</div><div><br></div><div>The log suggestst that the problem is the missing profile-dir in the ssh command:</div><div><br></div><div>
Starting SSHEngineLauncher: [&#39;ssh&#39;, &#39;-tt&#39;, u&#39;bergen&#39;, &#39;/opt/epd-7.1-2-rh5-x86_64/bin/python &#39;, &#39;-c &#39;, &#39;&quot;from IPython.parallel.apps.ipengineapp import launch_new_instance;   launch_new_instance()&quot;&#39;, &#39;--profile-dir&#39;, u&#39;&#39;, &#39;--log-to-file&#39;, &#39;--log-level=20&#39;]</div>
<div><br></div><div><br></div><div>Everything just falls apart after that. I don&#39;t really know how to go on about this, so any help would be very helpful!</div><div><br></div><div>Thanks!</div><div>Jose</div><div><br>
</div><div><br></div><div><br></div><div>################################################################################################################################################################</div><div><br></div>
<div># stdout from ipcluster start --profile=ssh --debug </div><div><br></div><div><div>2012-08-10 19:33:42,709.709 [IPClusterStart] Config changed:</div><div>2012-08-10 19:33:42,709.709 [IPClusterStart] {&#39;Application&#39;: {&#39;log_level&#39;: 10}, &#39;BaseIPythonApplication&#39;: {&#39;profile&#39;: u&#39;ssh&#39;}}</div>
<div>2012-08-10 19:33:42,777.777 [IPClusterStart] Using existing profile dir: u&#39;/home/x/.config/ipython/profile_ssh&#39;</div><div>2012-08-10 19:33:42,778.778 [IPClusterStart] Searching path [u&#39;/home/x&#39;, u&#39;/home/x/.config/ipython/profile_ssh&#39;] for config files</div>
<div>2012-08-10 19:33:42,778.778 [IPClusterStart] Attempting to load config file: ipython_config.py</div><div>2012-08-10 19:33:42,780.780 [IPClusterStart] Loaded config file: /home/x/.config/ipython/profile_ssh/ipython_config.py</div>
<div>2012-08-10 19:33:42,780.780 [IPClusterStart] Attempting to load config file: ipcluster_config.py</div><div>2012-08-10 19:33:42,783.783 [IPClusterStart] Loaded config file: /home/x/.config/ipython/profile_ssh/ipcluster_config.py</div>
<div>2012-08-10 19:33:42,784.784 [IPClusterStart] Config changed:</div><div>2012-08-10 19:33:42,784.784 [IPClusterStart] {&#39;BaseIPythonApplication&#39;: {&#39;profile&#39;: u&#39;ssh&#39;}, &#39;IPClusterEngines&#39;: {&#39;engine_launcher_class&#39;: &#39;SSHEngineSetLauncher&#39;}, &#39;Application&#39;: {&#39;log_level&#39;: 10}, &#39;ProfileDir&#39;: {}, &#39;SSHLauncher&#39;: {&#39;to_fetch&#39;: [], &#39;to_send&#39;: []}, &#39;SSHEngineSetLauncher&#39;: {&#39;remote_profile_dir&#39;: &#39;/home/x/.config/ipython/profile_ssh&#39;, &#39;engine_cmd&#39;: [&#39;/opt/epd-7.1-2-rh5-x86_64/bin/python &#39;, &#39;-c &#39;, &#39;&quot;from IPython.parallel.apps.ipengineapp import launch_new_instance;   launch_new_instance()&quot;&#39;], &#39;engines&#39;: {&#39;bergen&#39;: 1}}}</div>
<div>2012-08-10 19:33:42.788 [IPClusterStart] Starting ipcluster with [daemon=False]</div><div>2012-08-10 19:33:42.802 [IPClusterStart] Creating pid file: /home/x/.config/ipython/profile_ssh/pid/ipcluster.pid</div><div>2012-08-10 19:33:42.832 [IPClusterStart] Starting Controller with LocalControllerLauncher</div>
<div>2012-08-10 19:33:42.832 [IPClusterStart] Starting LocalControllerLauncher: [&#39;/opt/epd-7.1-2-rh5-x86_64/bin/python&#39;, &#39;-c&#39;, &#39;from IPython.parallel.apps.ipcontrollerapp import launch_new_instance; launch_new_instance()&#39;, &#39;--profile-dir&#39;, u&#39;/home/x/.config/ipython/profile_ssh&#39;, &#39;--cluster-id&#39;, u&#39;&#39;, &#39;--log-to-file&#39;, &#39;--log-level=20&#39;]</div>
<div>2012-08-10 19:33:42.841 [IPClusterStart] Process &#39;/opt/epd-7.1-2-rh5-x86_64/bin/python&#39; started: 16905</div><div>2012-08-10 19:33:43.546 [IPClusterStart] 2012-08-10 19:33:43,545.545 [IPControllerApp] Using existing profile dir: u&#39;/home/x/.config/ipython/profile_ssh&#39;</div>
<div>2012-08-10 19:33:43.789 [IPClusterStart] Starting 1 Engines with SSHEngineSetLauncher</div><div>2012-08-10 19:33:43.791 [IPClusterStart] Starting SSHEngineLauncher: [&#39;ssh&#39;, &#39;-tt&#39;, u&#39;bergen&#39;, &#39;/opt/epd-7.1-2-rh5-x86_64/bin/python &#39;, &#39;-c &#39;, &#39;&quot;from IPython.parallel.apps.ipengineapp import launch_new_instance;   launch_new_instance()&quot;&#39;, &#39;--profile-dir&#39;, u&#39;&#39;, &#39;--log-to-file&#39;, &#39;--log-level=20&#39;]</div>
<div>2012-08-10 19:33:43.801 [IPClusterStart] Process &#39;ssh&#39; started: 16920</div><div>2012-08-10 19:33:43.801 [IPClusterStart] Process &#39;engine set&#39; started: [None]</div><div>2012-08-10 19:33:43.892 [IPClusterStart] tcgetattr: Invalid argument</div>
<div>2012-08-10 19:33:43.996 [IPClusterStart] 2012-08-10 19:33:43.996 [scheduler] Scheduler started [leastload]</div><div>2012-08-10 19:33:45.133 [IPClusterStart] usage: -c [-h] [--ip ENGINEFACTORY.IP] [--work-dir IPENGINEAPP.WORK_DIR]</div>
<div>2012-08-10 19:33:45.133 [IPClusterStart]           [--file IPENGINEAPP.URL_FILE] [--port ENGINEFACTORY.REGPORT]</div><div>2012-08-10 19:33:45.133 [IPClusterStart]           [--profile-dir PROFILEDIR.LOCATION]</div><div>
2012-08-10 19:33:45.134 [IPClusterStart]           [--log-to-file [IPENGINEAPP.LOG_TO_FILE]]</div><div>2012-08-10 19:33:45.134 [IPClusterStart]           [--sshkey ENGINEFACTORY.SSHKEY] [--location ENGINEFACTORY.LOCATION]</div>
<div>2012-08-10 19:33:45.134 [IPClusterStart]           [--clean-logs IPENGINEAPP.CLEAN_LOGS]</div><div>2012-08-10 19:33:45.134 [IPClusterStart]           [--profile IPENGINEAPP.PROFILE]</div><div>2012-08-10 19:33:45.134 [IPClusterStart]           [--cluster-id IPENGINEAPP.CLUSTER_ID] [--mpi MPI.USE]</div>
<div>2012-08-10 19:33:45.134 [IPClusterStart]           [--ssh ENGINEFACTORY.SSHSERVER] [-c IPENGINEAPP.STARTUP_COMMAND]</div><div>2012-08-10 19:33:45.134 [IPClusterStart]           [--ident SESSION.SESSION] [--ipython-dir IPENGINEAPP.IPYTHON_DIR]</div>
<div>2012-08-10 19:33:45.134 [IPClusterStart]           [--url ENGINEFACTORY.URL] [--log-level IPENGINEAPP.LOG_LEVEL]</div><div>2012-08-10 19:33:45.135 [IPClusterStart]           [-s IPENGINEAPP.STARTUP_SCRIPT] [--log-url IPENGINEAPP.LOG_URL]</div>
<div>2012-08-10 19:33:45.135 [IPClusterStart]           [--timeout ENGINEFACTORY.TIMEOUT] [--keyfile SESSION.KEYFILE]</div><div>2012-08-10 19:33:45.135 [IPClusterStart]           [--transport ENGINEFACTORY.TRANSPORT] [--user SESSION.USERNAME]</div>
<div>2012-08-10 19:33:45.135 [IPClusterStart]           [--no-secure] [--secure] [--quiet] [--init] [--debug]</div><div>2012-08-10 19:33:45.135 [IPClusterStart] -c: error: argument --profile-dir: expected one argument</div>
<div>2012-08-10 19:33:45.162 [IPClusterStart] Connection to bergen closed.</div><div>2012-08-10 19:33:45.163 [IPClusterStart] Process &#39;ssh&#39; stopped: {&#39;pid&#39;: 16920, &#39;exit_code&#39;: 2}</div><div>2012-08-10 19:33:45.163 [IPClusterStart] Process &#39;engine set&#39; stopped: {&#39;bergen/0&#39;: {&#39;pid&#39;: 16920, &#39;exit_code&#39;: 2}}</div>
<div>2012-08-10 19:33:45.163 [IPClusterStart] </div><div>            Engines shutdown early, they probably failed to connect.</div><div>            </div><div>            Check the engine log files for output.</div><div>            </div>
<div>            If your controller and engines are not on the same machine, you probably</div><div>            have to instruct the controller to listen on an interface other than localhost.</div><div>            </div><div>
            You can set this by adding &quot;--ip=&#39;*&#39;&quot; to your ControllerLauncher.controller_args.</div><div>            </div><div>            Be sure to read our security docs before instructing your controller to listen on</div>
<div>            a public interface.</div><div>            </div><div>2012-08-10 19:33:45.164 [IPClusterStart] IPython cluster: stopping</div><div>2012-08-10 19:33:45.222 [IPClusterStart] Traceback (most recent call last):</div>
<div>2012-08-10 19:33:45.222 [IPClusterStart]   File &quot;&lt;string&gt;&quot;, line 1, in &lt;module&gt;</div><div>2012-08-10 19:33:45.222 [IPClusterStart]   File &quot;/home/x/.local/lib/python2.7/site-packages/IPython/parallel/apps/ipcontrollerapp.py&quot;, line 487, in launch_new_instance</div>
<div>2012-08-10 19:33:45.222 [IPClusterStart]     app.start()</div><div>2012-08-10 19:33:45.222 [IPClusterStart]   File &quot;/home/x/.local/lib/python2.7/site-packages/IPython/parallel/apps/ipcontrollerapp.py&quot;, line 461, in start</div>
<div>2012-08-10 19:33:45.223 [IPClusterStart]     self.factory.loop.start()</div><div>2012-08-10 19:33:45.223 [IPClusterStart]   File &quot;/opt/epd-7.1-2-rh5-x86_64/lib/python2.7/site-packages/zmq/eventloop/ioloop.py&quot;, line 251, in start</div>
<div>2012-08-10 19:33:45.223 [IPClusterStart]     event_pairs = self._impl.poll(poll_timeout)</div><div>2012-08-10 19:33:45.223 [IPClusterStart]   File &quot;poll.pyx&quot;, line 189, in zmq.core.poll.Poller.poll (zmq/core/poll.c:2120)</div>
<div>2012-08-10 19:33:45.223 [IPClusterStart]   File &quot;poll.pyx&quot;, line 101, in zmq.core.poll._poll (zmq/core/poll.c:1400)</div><div>2012-08-10 19:33:45.223 [IPClusterStart] ZMQError: Interrupted system call</div>
<div>2012-08-10 19:33:45.223 [IPClusterStart] </div><div>2012-08-10 19:33:45.224 [IPClusterStart] If you suspect this is an IPython bug, please report it at:</div><div>2012-08-10 19:33:45.224 [IPClusterStart]     <a href="https://github.com/ipython/ipython/issues">https://github.com/ipython/ipython/issues</a></div>
<div>2012-08-10 19:33:45.224 [IPClusterStart] or send an email to the mailing list at <a href="mailto:ipython-dev@scipy.org">ipython-dev@scipy.org</a></div><div>2012-08-10 19:33:45.224 [IPClusterStart] </div><div>2012-08-10 19:33:45.224 [IPClusterStart] You can print a more detailed traceback right now with &quot;%tb&quot;, or use &quot;%debug&quot;</div>
<div>2012-08-10 19:33:45.225 [IPClusterStart] to interactively debug it.</div><div>2012-08-10 19:33:45.225 [IPClusterStart] </div><div>2012-08-10 19:33:45.225 [IPClusterStart] Extra-detailed tracebacks for bug-reporting purposes can be enabled via:</div>
<div>2012-08-10 19:33:45.225 [IPClusterStart]     c.Application.verbose_crash=True</div><div>2012-08-10 19:33:45.225 [IPClusterStart] </div><div>2012-08-10 19:33:45.260 [IPClusterStart] Process &#39;/opt/epd-7.1-2-rh5-x86_64/bin/python&#39; stopped: {&#39;pid&#39;: 16905, &#39;exit_code&#39;: 1}</div>
<div>2012-08-10 19:33:48.164 [IPClusterStart] Removing pid file: /home/x/.config/ipython/profile_ssh/pid/ipcluster.pid</div></div><div><br></div><div><br></div>