[IPython-dev] new doc for parallel sessions on clusters

MinRK benjaminrk@gmail....
Tue Jun 21 11:41:25 CDT 2011


On Tue, Jun 21, 2011 at 09:31, Johann Cohen-Tanugi
<johann.cohentanugi@gmail.com> wrote:
> hmmm ok, after the merge is this the right way to get the code :
>  git clone git://github.com/ipython/ipython.git ipython
> I removed the whole source directory, and the local install stuff related to
> python, and I started from scratch using the git command above.
> then :
> python setup.py install
> --prefix=/afs/slac/g/glast/users/cohen/IPYDEV/local/which python
>
> perhaps the problem is that there is an ipython installed with the native
> python that I use above, and I am wrong to assume that having
> /afs/slac/g/glast/users/cohen/IPYDEV/local/.../site-packages at the top of
> PYTHONPATH is enough to ensure that it overrides the one installed with
> python?

If it was installed with setuptools/easy_install, yes - they inject
packages at the front sys.path at runtime, ending up ahead of
PYTHONPATH. easy_installed packages can even come before '.'.

You can check which one you are importing by looking at `IPython.__file__`.

It's possible that I've messed something up, but the source looks
right (https://github.com/ipython/ipython/blob/master/IPython/core/profileapp.py#L127)

>
> Anyway, I will continue investigating....
> sorry for the noise,
> JOhann
>
> On 06/21/2011 05:27 PM, MinRK wrote:
>>
>> On Tue, Jun 21, 2011 at 07:27, Johann Cohen-Tanugi
>> <johann.cohen-tanugi@lupm.univ-montp2.fr>  wrote:
>>>
>>> hi there, I tried to build the sphinx doc on my machine after a pull of
>>> the
>>> head (after Min announced the merge of newapp into master), and I see the
>>> following in docs/source/parallel/parallel_process.txt :
>>> 1/ A trivial typo at :
>>> :command:`ipcluster` has a notion of Launchers that can start controllers
>>> and engines with various remote execution schemes.  Currently supported
>>> models include :command:`ssh`, :command`mpiexec`, PBS-style (Torque,
>>> SGE),
>>> and Windows HPC Server.
>>>
>>>  |_____>
>>> should be :command:`mpiexec`
>>> 2/ I can read the following lines :
>>>    $ ipython profile create --parallel profile=mpi
>>> or
>>>    $ ipython profile create --parallel profile=pbs
>>>
>>> while ipython profile create does *not* accept a --parallel option
>>> AFAICT,
>>> but rather --cluster or --no-cluster.
>>
>> Then you do not have current master, because the flag is indeed
>> '--parallel'.
>>
>>> best,
>>> Johann
>>>
>>> On 06/21/2011 07:37 AM, Min RK wrote:
>>>>
>>>> Yes, it is going to be standard for the cluster profile and interactive
>>>> IPython to be different - the profile for the Client is principally for
>>>> connection info, and the shell profile is for configuring your
>>>> interactive
>>>> environment.  There's no reason to change your interactive config just
>>>> to
>>>> connect to a different cluster.
>>>>
>>>> That said, the default profile of the Client should probably be that of
>>>> the current application, not just 'default'.
>>>>
>>>> -MinRK
>>>>
>>>> On Jun 20, 2011, at 22:11, Johann
>>>> Cohen-Tanugi<johann.cohen-tanugi@lupm.univ-montp2.fr>    wrote:
>>>>
>>>>> hi- I don't think we should print the profile name in the default case,
>>>>>>
>>>>>> it's just noise.  I realize we now have a more consistent structure
>>>>>> for profiles and even the default case is now a profile, but we should
>>>>>> keep the amount of printed stuff to a minimum in the default cases.
>>>>>>
>>>>> Actually I have a question here : I was trying newapp, following Min's
>>>>> advice, to try to add the LSF support in parallel.apps. From what I
>>>>> could
>>>>> gather
>>>>> I did
>>>>> ipcluster start -p lsf -n 2
>>>>> which created profile_lsf in my $HOME/.ipython directory, but then when
>>>>> I
>>>>> started another terminal window for the ipython session, I typed
>>>>> ipython profile=lsf
>>>>> and this loaded the default profile, so that I had to type :
>>>>>
>>>>> from IPython.parallel import Client
>>>>> c = Client(profile='lsf')
>>>>>
>>>>> so that unless this is a bug or an operator mistake, there seems to be
>>>>> 2
>>>>> 'profiles' in such a use case : the ipython global one, and the
>>>>> parallel lsf
>>>>> one. I find that a bit confusing, and maybe there is a way to merge the
>>>>> 2?
>>>>>
>>>>> best,
>>>>> Johann
>


More information about the IPython-dev mailing list