[IPython-dev] new doc for parallel sessions on clusters
Tue Jun 21 10:27:35 CDT 2011
On Tue, Jun 21, 2011 at 07:27, Johann Cohen-Tanugi
> hi there, I tried to build the sphinx doc on my machine after a pull of the
> head (after Min announced the merge of newapp into master), and I see the
> following in docs/source/parallel/parallel_process.txt :
> 1/ A trivial typo at :
> :command:`ipcluster` has a notion of Launchers that can start controllers
> and engines with various remote execution schemes. Currently supported
> models include :command:`ssh`, :command`mpiexec`, PBS-style (Torque, SGE),
> and Windows HPC Server.
> should be :command:`mpiexec`
> 2/ I can read the following lines :
> $ ipython profile create --parallel profile=mpi
> $ ipython profile create --parallel profile=pbs
> while ipython profile create does *not* accept a --parallel option AFAICT,
> but rather --cluster or --no-cluster.
Then you do not have current master, because the flag is indeed '--parallel'.
> On 06/21/2011 07:37 AM, Min RK wrote:
>> Yes, it is going to be standard for the cluster profile and interactive
>> IPython to be different - the profile for the Client is principally for
>> connection info, and the shell profile is for configuring your interactive
>> environment. There's no reason to change your interactive config just to
>> connect to a different cluster.
>> That said, the default profile of the Client should probably be that of
>> the current application, not just 'default'.
>> On Jun 20, 2011, at 22:11, Johann
>> Cohen-Tanugi<firstname.lastname@example.org> wrote:
>>> hi- I don't think we should print the profile name in the default case,
>>>> it's just noise. I realize we now have a more consistent structure
>>>> for profiles and even the default case is now a profile, but we should
>>>> keep the amount of printed stuff to a minimum in the default cases.
>>> Actually I have a question here : I was trying newapp, following Min's
>>> advice, to try to add the LSF support in parallel.apps. From what I could
>>> I did
>>> ipcluster start -p lsf -n 2
>>> which created profile_lsf in my $HOME/.ipython directory, but then when I
>>> started another terminal window for the ipython session, I typed
>>> ipython profile=lsf
>>> and this loaded the default profile, so that I had to type :
>>> from IPython.parallel import Client
>>> c = Client(profile='lsf')
>>> so that unless this is a bug or an operator mistake, there seems to be 2
>>> 'profiles' in such a use case : the ipython global one, and the parallel lsf
>>> one. I find that a bit confusing, and maybe there is a way to merge the 2?
More information about the IPython-dev