[IPython-User] Can't get ipcluster to run on LAN
Wed Aug 8 08:57:26 CDT 2012
On Wednesday 08 August 2012 09:42:35 Johann Rohwer wrote:
> On Tuesday 07 August 2012 16:03:52 MinRK wrote:
> > Try removing the old connection files (in `ipython locate profile
> > home_ssh`/security).
> > There might be an issue when reuse_files=True, that sometimes the
> > old
> > connection files can set the config, so if you ran once with
> > ip=127.0.0.1, then set reuse_files=True, it's possible that
> > 127.0.0.1 is still being used.
> > Can you verify the contents of the JSON files?
I think I've finally come to the bottom of this. I've experienced
similar problems in setting up ipcluster at work (12 nodes linux SSH
cluster with shared home directory).
1. Downgrading to IPython 0.12.1, everything works fine, also with the
default configuration, for the SSH Launcher.
2. It turns out that there is a problem with option parsing. When I
run "ipcluster start --ip='*'" just with the default profile on
localhost, a directory '*' with inside the profile_default/...
hierarchy is created in the cwd. Similarly when I run "ipcluster start
--ip=184.108.40.206'' a directory "220.127.116.11" is created in the cwd with the
profile_default hierarchy inside.
So it seems that this --ip argument is interpreted as --profile_dir or
similar. The config setting c.HubFactory.ip has a similar effect.
The cluster still starts on localhost as the engines can read the json
files from the "wrong" directory, but this obviously all fails when the
files are to be created on remote machines via SSH as a dir like /* or
/18.104.22.168 cannot be created on a remote machine (no permission).
I've tried diffing the 0.12.1 code and 0.13 code to try find what's going
on. In IPython/parallel/apps/launcher.py the class SSHClusterLauncher
has a method _remote_profie_dir_default(self) (line 653), that does
some mangling of the profile dir variable. NB spelling mistake in the
method name. This method is not present in 0.12.1 AFAICT.
But this can't be the whole story as the error also occurs in a
default setup just on localhost, which does not call the SSH Launcher.
Clearly it also has to do with option parsing. I am really not at all
familiar with the IPython code to investigate this further without
guidance, but will gladly test any fixes. As said, 0.12.1 works "out of
the box" as intended.
Hierdie e-pos mag vertroulike inligting bevat en mag regtens geprivilegeerd wees en is slegs bedoel vir die persoon aan wie dit geadresseer is. Indien u nie die bedoelde ontvanger is nie, word u hiermee in kennis gestel dat u hierdie dokument geensins mag gebruik, versprei of kopieer nie. Stel ook asseblief die sender onmiddellik per telefoon in kennis en vee die e-pos uit. Die Universiteit aanvaar nie aanspreeklikheid vir enige skade, verlies of uitgawe wat voortspruit uit hierdie e-pos en/of die oopmaak van enige lÃªs aangeheg by hierdie e-pos nie.
This e-mail may contain confidential information and may be legally privileged and is intended only for the person to whom it is addressed. If you are not the intended recipient, you are notified that you may not use, distribute or copy this document in any manner whatsoever. Kindly also notify the sender immediately by telephone, and delete the e-mail. The University does not accept liability for any damage, loss or expense arising from this e-mail and/or accessing any files attached to this e-mail.
More information about the IPython-User