[IPython-User] launching engines via python

Ben Edwards BJEdwards@gmail....
Tue Sep 25 16:39:27 CDT 2012

I ran into this problem not too long ago. I don't think there is any
programmatic way to do this within iPython, but you can use subprocess.
I've added in a signal handler, so if I have to stop my script in the
middle of a run it automatically shuts down the cluster.

import subprocess
> import signal
> import time

def signal_handler(signal,frame):
>      print("Shutting Down Cluster")
>      subprocess.Popen(["ipcluster", "stop", "--quiet"])
>      sys.exit(0)

> signal.signal(signal.SIGINT, signal_handler)
> subprocess.Popen(["ipcluster",
>                   "start",
>                   "--n=" + str(12),
>                   "--daemonize",
>                   "--quiet"])
> time.sleep(5) #time for the cluster to spin up
> #Code using cluster
> subprocess.Popen(["ipcluster", "stop", "--quiet"])

Hope this helps.


On Tue, Sep 25, 2012 at 1:13 PM, Brent Pedersen <bpederse@gmail.com> wrote:

> Hi,
> I'm attempting to understand how I can use an ipengine on an LSF cluster.
> I've done something like this:
>      ipython profile create --parallel --profile=lsf
>      # edit .ipython/profile_lsf/ipcluster_config.py to use LSF
>      ipcluster start --profile=lsf --daemonize
> But, I'd like to be able to start the engines programatically.
> After the above command, I can do this:
>     c = Client(profile='lsf')
> Is there a way to "connect" to the profile and have it dynamically launch
> engines via python?
> I found the IPEngineApp, but it's not clear to me how to use that.
> Is there an example somewhere that launches engines via the API, rather
> than
> the command line? Even one for PBS would be helpful.
> thanks,
> -Brent
> _______________________________________________
> IPython-User mailing list
> IPython-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-user
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20120925/4e3a59c3/attachment.html 

More information about the IPython-User mailing list