[IPython-User] launching engines via python

MinRK benjaminrk@gmail....
Thu Sep 27 18:40:03 CDT 2012


You actually can start/stop/manage engine and/or controller subprocesses
with the same Launchers used by ipcontroller.

The interface is not really meant for interactive use, so it's not awesome,
but it does work:

http://nbviewer.ipython.org/3797107


On Tue, Sep 25, 2012 at 2:39 PM, Ben Edwards <BJEdwards@gmail.com> wrote:

> I ran into this problem not too long ago. I don't think there is any
> programmatic way to do this within iPython, but you can use subprocess.
> I've added in a signal handler, so if I have to stop my script in the
> middle of a run it automatically shuts down the cluster.
>
> import subprocess
>> import signal
>> import time
>
>
> def signal_handler(signal,frame):
>>      print("Shutting Down Cluster")
>>      subprocess.Popen(["ipcluster", "stop", "--quiet"])
>>      sys.exit(0)
>>
>
>
>> signal.signal(signal.SIGINT, signal_handler)
>> subprocess.Popen(["ipcluster",
>>                   "start",
>>                   "--n=" + str(12),
>>                   "--daemonize",
>>                   "--quiet"])
>> time.sleep(5) #time for the cluster to spin up
>> #Code using cluster
>> subprocess.Popen(["ipcluster", "stop", "--quiet"])
>
>
> Hope this helps.
>
> Ben
>
> On Tue, Sep 25, 2012 at 1:13 PM, Brent Pedersen <bpederse@gmail.com>wrote:
>
>> Hi,
>> I'm attempting to understand how I can use an ipengine on an LSF cluster.
>> I've done something like this:
>>
>>      ipython profile create --parallel --profile=lsf
>>      # edit .ipython/profile_lsf/ipcluster_config.py to use LSF
>>      ipcluster start --profile=lsf --daemonize
>>
>> But, I'd like to be able to start the engines programatically.
>> After the above command, I can do this:
>>
>>     c = Client(profile='lsf')
>>
>> Is there a way to "connect" to the profile and have it dynamically launch
>> engines via python?
>> I found the IPEngineApp, but it's not clear to me how to use that.
>>
>> Is there an example somewhere that launches engines via the API, rather
>> than
>> the command line? Even one for PBS would be helpful.
>>
>> thanks,
>> -Brent
>> _______________________________________________
>> IPython-User mailing list
>> IPython-User@scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-user
>>
>
>
> _______________________________________________
> IPython-User mailing list
> IPython-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-user
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20120927/8f4f1f79/attachment.html 


More information about the IPython-User mailing list