[IPython-User] Parallel Setup: Can connect engines over network, but not run Hello World

Markus Baden markus.baden@gmail....
Thu Oct 6 01:31:41 CDT 2011


Hi list,

I'm trying to use IPython together with our High Performance Computing
facilties at NUS. The jobs are distributed via LSF, so I intend to follow
the discussion on setting up IPython with LSF [1]. Before using the LSF
setup I wanted to test things by just having remote engines on their
computer, controlled by a local controller on my laptop.

However, I have the problem that there is no shared network drive between
the computers that are used for submitting the jobs (which would run the
engines) and my laptop. So far I was able to get a local ipcontroller
running and remote ipengines that show up as connected. However
c[:].apply_snyc(lambda: 'Hello World') did not work.

Here is what I did:

- ipython profile create --parallel  --profile=hpc_test
- set c.HubFactory.ip = '*' in ipcontroller_config.py
- run on laptop ipcontroller --profile=hpc_test&
- create hpc_test profile on remote computer
- transfer ipcontroller-engine.json from laptop to remote computer
- run ipengine --profile=hpc_test on remote computer, which led the
ipcontroller on my laptop to acknowledge that an engine was connected
- on local machine start ipython, c = Client(profile='hpc_test')
- c.ids shows the correct ids, but c[:].apply_snyc(lambda: 'Hello World')
just hangs

Since I am totally new to this, I'm super confused and don't know where to
proceed from here. Somehow I was expecting that I have to tell the engine
over which protocol to talk to the controller. I was intending to use ssh
for that, but somehow I could connect engines without even specifying this.

Could somebody point me in the right direction, on where to proceed from
here on? Any hints are greatly appreciated.

Thanks a lot,

Markus

[1] http://thread.gmane.org/gmane.comp.python.ipython.user/6487
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20111006/ad986a85/attachment.html 


More information about the IPython-User mailing list