[IPython-User] ipcluster, profiles, startup scripts and the web notebook
Fri Jul 27 16:31:33 CDT 2012
On Fri, Jul 27, 2012 at 2:19 PM, Matthew Turk <firstname.lastname@example.org> wrote:
> Hi Min,
> On Fri, Jul 27, 2012 at 4:04 PM, MinRK <email@example.com> wrote:
> > On Fri, Jul 27, 2012 at 10:53 AM, Matthew Turk <firstname.lastname@example.org>
> > wrote:
> >> Hi there,
> >> I'm trying to set up a custom profile for IPython running on my local
> >> machine under MPI, but I'm running into trouble. This is with the
> >> released 0.13. I'd like at startup for a couple commands to be run,
> >> to signal to my Python package that it should expect to be running in
> >> parallel (with MPI). Unfortunately, I'm not quite getting the
> >> behavior I'd expect, and I think it must be something simple on my end
> >> that I'm not understanding.
> >> I've created a new profile with:
> >> ipython profile create --profile yt --parallel
> >> Into the startup directory I've placed the normal startup commands --
> >> in this case, the main import for the module I intend to work with --
> >> inside a 00-startup.py file. I'm able to spawn new notebooks that
> >> start up as I'd expect if I do "ipython notebook --profile=yt". So
> >> that seems to work properly. For starting in parallel, I've modified
> >> the generated profile in a few places:
> >> * I've set c.IPClusterEngines.profile = u'yt' in ipcluster_config.py
> >> * I've changed c.IPClusterEngines.engine_launcher_class to "MPI" in
> >> ipcluster_config.py
> > Exactly right.
> >> * In ipengine_config.py I've set c.IPEngineApp.profile = u'yt' (I was
> >> not sure if this was either unnecessary or redundant)
> > Yes, setting profile in config files is indeed always redundant (both
> > here).
> >> * In ipengine_config.py I've added a startup command in
> >> c.IPEngineApp.startup_command
> > Re-reading the code, this config was broken when the forked Kernel
> > implementations were merged for 0.13.
> > I think right now it is impossible to trigger startup code on engines in
> > 0.13 or master.
> > Can you open an Issue for this?
> > The reason being that Kernels/Shells are shared, but the Application
> > are still separate, and that is where this startup config lives.
> > Ultimately, IPEngineApp should go away, but we aren't quite there yet.
> Ah, okay. Thanks for digging into this. I've opened issue #2213:
Thanks! And I may have a fix
> >> In the web notebook launcher, changes to the number of processors is
> >> reflected in what happens when I click on "start". And looking at the
> >> output of top/ps I also see that the engines have been launched with
> >> mpiexec as I have requested. I'm also able to connect to them and
> >> verify that they have been launched, but running "dir()" doesn't show
> >> the expected imports, from the startup_command. I'm also unable to
> >> get anything to be run if I set c.MPI.init_script to be an existing
> >> script. Am I missing something with how to set up the configuration?
> >> Thanks for any ideas,
> >> Matt
> >> _______________________________________________
> >> IPython-User mailing list
> >> IPython-User@scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-user
> > _______________________________________________
> > IPython-User mailing list
> > IPython-User@scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-user
> IPython-User mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the IPython-User