[IPython-User] launching engines via python

[Ben Edwards]

I ran into this problem not too long ago. I don't think there is any
programmatic way to do this within iPython, but you can use subprocess.
I've added in a signal handler, so if I have to stop my script in the
middle of a run it automatically shuts down the cluster.

import subprocess
> import signal
> import time


def signal_handler(signal,frame):
>      print("Shutting Down Cluster")
>      subprocess.Popen(["ipcluster", "stop", "--quiet"])
>      sys.exit(0)
>


> signal.signal(signal.SIGINT, signal_handler)
> subprocess.Popen(["ipcluster",
>                   "start",
>                   "--n=" + str(12),
>                   "--daemonize",
>                   "--quiet"])
> time.sleep(5) #time for the cluster to spin up
> #Code using cluster
> subprocess.Popen(["ipcluster", "stop", "--quiet"])


Hope this helps.

On Tue, Sep 25, 2012 at 1:13 PM, Brent Pedersen <bpederse@gmail.com> wrote:

> Hi,
> I'm attempting to understand how I can use an ipengine on an LSF cluster.
> I've done something like this:
>
>      ipython profile create --parallel --profile=lsf
>      # edit .ipython/profile_lsf/ipcluster_config.py to use LSF
>      ipcluster start --profile=lsf --daemonize
>
> But, I'd like to be able to start the engines programatically.
> After the above command, I can do this:
>
>     c = Client(profile='lsf')
>
> Is there a way to "connect" to the profile and have it dynamically launch
> engines via python?
> I found the IPEngineApp, but it's not clear to me how to use that.
>
> Is there an example somewhere that launches engines via the API, rather
> than
> the command line? Even one for PBS would be helpful.
>
> thanks,
> -Brent
> _______________________________________________
> IPython-User mailing list
> IPython-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20120925/4c83ff58/attachment.html