[IPython-dev] ipcontroller & ipengine

Brian Granger ellisonbg.net@gmail....
Wed Sep 3 15:43:15 CDT 2008


> I'm trying ipython's parallel capabilities once again, but things have
> changed... a lot apparently, since I last tried...

Yep, that is very true.  Are you using a 0.9 beta or release candidate?

> I used to start 'ipcontroller --engine-ip 10.0.0.211' on localhost, and
> then 'ipengine -n8' on 10.0.0.211 (an 8-core machine). I could then
> connect from localhost to the controller, which connected to the
> engines@10.0.0.211. Or I started ipcluster -n8 on 10.0.0.211, and then
> started a client connection from ipython specifying 10.0.0.211 as the
> controller-host to connect to.
> Neither works anymore.

Yep, now you have to fiddle with furl files.  While this may seem like
a pain, it is actually a huge improvement in usability.

> I know I can fiddle with furl files, but it is not convenient for me to
> have to copy a furl file each time I (re)start a cluster. I don't want
> to mess with ssh-agent either, if possible.

Furl files are persistent.  You only need to move them once and then
future controllers and engines will use those.  Let us know if this
doesn't work - it would be a bug.

> What would be the simplest way to achieve a situation where 1) the
> engines run on remotehost, 2) I can connect from kernel.client to those
> engines? Controller might run anywhere it is convenient...

1.  Start the controller somewhere.

2.  Copy ~/.ipython/ipcontroller-engine.furl to the machines where the
engine will run and ipcontroller-mec.furl and ipcontroller-tc.furl to
where the client will run.

3.  Start the engines using

ipengine --furl-file=FURL_FILE_LOCATION

Or if you just put ipcontroller-engine.furl in ~/.ipython on the
engine's host, you can leave --furl-file out.

4.  Start the client, passing to it, the furl file location.

The important point is that once you do (2), it never has to be done again!

Does that make sense?


> Btw this is the error when I try to start a controller on localhost,
> ready to listen for engines on 10.0.0.211:
>
> $  ipcontroller --engine-ip 10.0.0.211

--engine-ip is not a valid option anymore.

Let us know if this helps or if you have other questions.

Cheers,

Brian
> 2008-08-22 15:49:19+0200 [-] Log opened.
> 2008-08-22 15:49:19+0200 [-] foolscap.pb.Listener starting on 45401
> 2008-08-22 15:49:19+0200 [-] Traceback (most recent call last):
> 2008-08-22 15:49:19+0200 [-]   File "/usr/bin/ipcontroller", line 19, in
> <module>
> 2008-08-22 15:49:19+0200 [-]     ipcontroller.main()
> 2008-08-22 15:49:19+0200 [-]   File
> "/usr/lib/python2.5/site-packages/IPython/kernel/scripts/ipcontroller.py",
> line 363, in main
> 2008-08-22 15:49:19+0200 [-]     start_controller()
> 2008-08-22 15:49:19+0200 [-]   File
> "/usr/lib/python2.5/site-packages/IPython/kernel/scripts/ipcontroller.py",
> line 217, in start_controller
> 2008-08-22 15:49:19+0200 [-]     main_service.startService()
> 2008-08-22 15:49:19+0200 [-]   File
> "/usr/lib/python2.5/site-packages/twisted/application/service.py", line
> 260, in startService
> 2008-08-22 15:49:19+0200 [-]     service.startService()
> 2008-08-22 15:49:19+0200 [-]   File
> "build/bdist.linux-x86_64/egg/foolscap/pb.py", line 591, in startService
> 2008-08-22 15:49:19+0200 [-]   File
> "/usr/lib/python2.5/site-packages/twisted/application/service.py", line
> 260, in startService
> 2008-08-22 15:49:19+0200 [-]     service.startService()
> 2008-08-22 15:49:19+0200 [-]   File
> "/usr/lib/python2.5/site-packages/twisted/application/internet.py", line
> 91, in startService
> 2008-08-22 15:49:19+0200 [-]     self._port = self._getPort()
> 2008-08-22 15:49:19+0200 [-]   File
> "/usr/lib/python2.5/site-packages/twisted/application/internet.py", line
> 116, in _getPort
> 2008-08-22 15:49:19+0200 [-]     *self.args, **self.kwargs)
> 2008-08-22 15:49:19+0200 [-]   File
> "/usr/lib/python2.5/site-packages/twisted/internet/posixbase.py", line
> 328, in listenTCP
> 2008-08-22 15:49:19+0200 [-]     p.startListening()
> 2008-08-22 15:49:19+0200 [-]   File
> "/usr/lib/python2.5/site-packages/twisted/internet/tcp.py", line 739, in
> startListening
> 2008-08-22 15:49:19+0200 [-]     raise CannotListenError,
> (self.interface, self.port, le)
> 2008-08-22 15:49:19+0200 [-] twisted.internet.error.CannotListenError:
> Couldn't listen on 10.0.0.211:0: (99, 'Cannot assign requested address').
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev@scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


More information about the IPython-dev mailing list