[IPython-User] notebooks and clusters

Darren Govoni darren@ontrenet....
Mon Jul 23 19:15:08 CDT 2012


I see. Well, what I notice is that the notebook server creates the
config files when the cluster is started and deletes it when it stops.
They don't exist when the notebook server is run, only after a cluster
is started.

How do I control that? It's using an incorrect url: property preventing
remote engines from connecting (i.e. the notebook server
is not listening on the correct IP).

Also, if I do not supply --ip=* as in "ipython notebook --ip=*" then I
cannot access the notebook remotely. So that option is required for me.
However, even with this setting the url in the config is incorrect for
remote access.


On Mon, 2012-07-23 at 17:09 -0700, Min RK wrote:
> Command-line options to the notebook server have exactly zero effect
> on the cluster config. You *must* use the config files to configure
> the cluster.
> 
> -MinRK
> 
> On Jul 23, 2012, at 16:29, Darren Govoni <darren@ontrenet.com> wrote:
> 
> 
> 
> > Hi,
> >   I am trying to connect a remote engine to a running notebook using
> > the default config. I copied the file.
> > It contains.
> > {
> >   "url": "tcp://127.0.0.1:57726", 
> >   "exec_key": "6dac7baf-45c3-4ab2-b642-bfa65fdac0e9", 
> >   "ssh": "", 
> >   "location": "10.32.6.3"
> > }
> > 
> > 
> > The problem is that the url property is not using the --ip= address
> > I invoke the notebook with and
> > the remote engine says its timing out trying to register. Any clues?
> > 
> > thanks.
> > 
> > On Mon, 2012-07-23 at 13:52 -0700, MinRK wrote: 
> > 
> > > On Mon, Jul 23, 2012 at 1:38 PM, Darren Govoni <darren@ontrenet.com> wrote:
> > > > Ok, thanks.
> > > >
> > > > What I want to do is run remote engines manually and have them connect to
> > > > the notebooks controller.
> > > >
> > > > I assume the notebook as a config file it creates that I can use on remote
> > > > servers for engines to connect?
> > > 
> > > There is no notebook-specific config to the clusters.  Clicking the
> > > `start` button for profile 'foo' should be identical to doing
> > > `ipcluster start --profile foo`.  All of the parallel
> > > connection/configuration docs still apply.  That means that the
> > > controller creates `ipcontroller-{client|engine}.json` files in
> > > profile/security, and you can start engines with `ipengine
> > > --profile=foo` or `ipcluster engines --profile=foo` to connect to it.
> > > If your machines are not on a shared filesystem, you will need to pass
> > > the connection files around as described in the docs.
> > > 
> > > 
> > > 
> > > >
> > > > Or does it just use the default config location (e.g. user directory)?
> > > >
> > > >
> > > > On Mon, 2012-07-23 at 13:37 -0700, MinRK wrote:
> > > >
> > > > On Mon, Jul 23, 2012 at 1:25 PM, Darren Govoni <darren@ontrenet.com> wrote:
> > > >> Hi,
> > > >>   Can the notebook cluster connect with remote engines or vice-versa?
> > > >
> > > > Yes - the notebook cluster panel is just a very thin wrapper for
> > > > ipcluster.  So if you create a profile that starts engines via
> > > > PBS/SGE/SSH, etc. that will happen when you start a cluster from the
> > > > panel.  Similarly, you can start a cluster from the panel and add
> > > > engines to it with `ipengine` and/or `ipcluster engines` in all the
> > > > usual ways.  There is nothing special about clusters started from the
> > > > notebook panel, other than that they can also be stopped from there.
> > > >
> > > > The one thing you cannot do yet is actually edit the config files for
> > > > those profiles from the notebook interface - for that, you need access
> > > > to the filesystem, or you can do a dirty version by writing the the
> > > > config files with the `%%file` magic in a notebook.
> > > >
> > > > -MinRK
> > > >
> > > >>
> > > >> thanks.
> > > >>
> > > >> _______________________________________________
> > > >> IPython-User mailing list
> > > >> IPython-User@scipy.org
> > > >> http://mail.scipy.org/mailman/listinfo/ipython-user
> > > >>
> > > > _______________________________________________
> > > > IPython-User mailing list
> > > > IPython-User@scipy.org
> > > > http://mail.scipy.org/mailman/listinfo/ipython-user
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > IPython-User mailing list
> > > > IPython-User@scipy.org
> > > > http://mail.scipy.org/mailman/listinfo/ipython-user
> > > >
> > > _______________________________________________
> > > IPython-User mailing list
> > > IPython-User@scipy.org
> > > http://mail.scipy.org/mailman/listinfo/ipython-user
> > 
> > 
> > 
> > _______________________________________________
> > IPython-User mailing list
> > IPython-User@scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-user
> > 
> 
> _______________________________________________
> IPython-User mailing list
> IPython-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-user


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20120723/143c54e2/attachment.html 


More information about the IPython-User mailing list