[IPython-User] ipcluster, profiles, startup scripts and the web notebook

Matthew Turk matthewturk@gmail....
Fri Jul 27 12:53:04 CDT 2012


Hi there,

I'm trying to set up a custom profile for IPython running on my local
machine under MPI, but I'm running into trouble.  This is with the
released 0.13.  I'd like at startup for a couple commands to be run,
to signal to my Python package that it should expect to be running in
parallel (with MPI).  Unfortunately, I'm not quite getting the
behavior I'd expect, and I think it must be something simple on my end
that I'm not understanding.

I've created a new profile with:

ipython profile create --profile yt --parallel

Into the startup directory I've placed the normal startup commands --
in this case, the main import for the module I intend to work with --
inside a 00-startup.py file.  I'm able to spawn new notebooks that
start up as I'd expect if I do "ipython notebook --profile=yt".  So
that seems to work properly.  For starting in parallel, I've modified
the generated profile in a few places:

 * I've set c.IPClusterEngines.profile = u'yt' in ipcluster_config.py
 * I've changed c.IPClusterEngines.engine_launcher_class to "MPI" in
ipcluster_config.py
 * In ipengine_config.py I've set c.IPEngineApp.profile = u'yt' (I was
not sure if this was either unnecessary or redundant)
 * In ipengine_config.py I've added a startup command in
c.IPEngineApp.startup_command

In the web notebook launcher, changes to the number of processors is
reflected in what happens when I click on "start".  And looking at the
output of top/ps I also see that the engines have been launched with
mpiexec as I have requested.  I'm also able to connect to them and
verify that they have been launched, but running "dir()" doesn't show
the expected imports, from the startup_command.  I'm also unable to
get anything to be run if I set c.MPI.init_script to be an existing
script.  Am I missing something with how to set up the configuration?

Thanks for any ideas,

Matt


More information about the IPython-User mailing list