[SciPy-user] running scipy code simultaneously on several machines

Fernando Perez fperez.net@gmail....
Sun Oct 14 15:54:38 CDT 2007


On 10/14/07, Kurt Smith <kwmsmith@gmail.com> wrote:
> On 10/14/07, Brian Granger <ellisonbg.net@gmail.com> wrote:
> > Robert gives the link to more details about how to start everything.
> > But if you are just running on a single machine, the easiest way is to
> > just do:
> >
> > $ ipcluster -n 4
> >
> > >From the shell after installing ipython1.  Let us know if you have
> > further problems.
> >
> > Brian
>
> Thanks for the pointers!  Very impressive piece of work -- it will
> help me very much with my batch jobs and I'll be watching it as it
> matures.  Impressive scatter/gather interface, too -- very clean.
>
> There was a bit of a hangup when running ipcluster -- on my MacBook,
> if you put it in the background right away, the process is stopped
> immediately before the script has a chance to set up the controller
> and spawn the engines:
>
> $ ipcluster -n 4 &
> [1] 450
> $ Starting controller: Controller PID: 452
> [1]+  Stopped                 ipcluster -n 4
> $
>
> Since it is stopped and the engines aren't created, trying to use it fails.
>
> I have to run 'ipcluster -n <num>' in the foreground, let it set up
> the engines, suspend it (CTRL-Z) and put it in the background for
> things to work nicely.  Is there any way around this?  Not a big deal,
> just wondering if I'm doing anything wrong.

I suspect the problem is that it's trying to print out to stdout,
though on linux it can be backgrounded immediately (but I've seen
similarly other things that complain because they can't print to the
TTY, I don't know what determines whether the shell stops the process
or not when it tries to print from the background).

Since the ipcluster script is a convenient one to later stop via
Ctrl-C for cleanup, and waiting to see that the engines are all up and
running is also a good way things are ready to start working, I
typically just keep an open terminal for that one guy and leave it
open without backgrounding it.  But the whole process startup system
is being reworked into something more modular, so we should probably
add a 'quiet' mode for it that can be immediately backgrounded without
any more fuss, since it's also a valid use case.

Cheers,

f

ps - for further questions, I'd encourage to ask us on the ipython
list, so we don't clutter the scipy one with matters specific to
ipython that others here may not necessarily be interested in.


More information about the SciPy-user mailing list