[IPython-User] ipcluster, profiles, startup scripts and the web notebook

MinRK benjaminrk@gmail....
Fri Jul 27 15:04:17 CDT 2012


On Fri, Jul 27, 2012 at 10:53 AM, Matthew Turk <matthewturk@gmail.com>wrote:

> Hi there,
>
> I'm trying to set up a custom profile for IPython running on my local
> machine under MPI, but I'm running into trouble.  This is with the
> released 0.13.  I'd like at startup for a couple commands to be run,
> to signal to my Python package that it should expect to be running in
> parallel (with MPI).  Unfortunately, I'm not quite getting the
> behavior I'd expect, and I think it must be something simple on my end
> that I'm not understanding.
>
> I've created a new profile with:
>
> ipython profile create --profile yt --parallel
>
> Into the startup directory I've placed the normal startup commands --
> in this case, the main import for the module I intend to work with --
> inside a 00-startup.py file.  I'm able to spawn new notebooks that
> start up as I'd expect if I do "ipython notebook --profile=yt".  So
> that seems to work properly.  For starting in parallel, I've modified
> the generated profile in a few places:
>
>  * I've set c.IPClusterEngines.profile = u'yt' in ipcluster_config.py
>  * I've changed c.IPClusterEngines.engine_launcher_class to "MPI" in
> ipcluster_config.py
>

Exactly right.


>  * In ipengine_config.py I've set c.IPEngineApp.profile = u'yt' (I was
> not sure if this was either unnecessary or redundant)
>

Yes, setting profile in config files is indeed always redundant (both
cases, here).


>  * In ipengine_config.py I've added a startup command in
> c.IPEngineApp.startup_command
>

Re-reading the code, this config was broken when the forked Kernel
implementations were merged for 0.13.
I think right now it is impossible to trigger startup code on engines in
0.13 or master.
Can you open an Issue <https://github.com/ipython/ipython/issues/new> for
this?

The reason being that Kernels/Shells are shared, but the Application
objects are still separate, and that is where this startup config lives.
 Ultimately, IPEngineApp should go away, but we aren't quite there yet.


> In the web notebook launcher, changes to the number of processors is
> reflected in what happens when I click on "start".  And looking at the
> output of top/ps I also see that the engines have been launched with
> mpiexec as I have requested.  I'm also able to connect to them and
> verify that they have been launched, but running "dir()" doesn't show
> the expected imports, from the startup_command.  I'm also unable to
> get anything to be run if I set c.MPI.init_script to be an existing
> script.  Am I missing something with how to set up the configuration?
>
> Thanks for any ideas,
>
> Matt
> _______________________________________________
> IPython-User mailing list
> IPython-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20120727/725acc84/attachment.html 


More information about the IPython-User mailing list