[IPython-User] launching engines via python

[Brent Pedersen]

On Thu, Sep 27, 2012 at 5:40 PM, MinRK <benjaminrk@gmail.com> wrote:
> You actually can start/stop/manage engine and/or controller subprocesses
> with the same Launchers used by ipcontroller.
>
> The interface is not really meant for interactive use, so it's not awesome,
> but it does work:
>
> http://nbviewer.ipython.org/3797107
>

That's what I was looking for, thanks!

>
> On Tue, Sep 25, 2012 at 2:39 PM, Ben Edwards <BJEdwards@gmail.com> wrote:
>>
>> I ran into this problem not too long ago. I don't think there is any
>> programmatic way to do this within iPython, but you can use subprocess. I've
>> added in a signal handler, so if I have to stop my script in the middle of a
>> run it automatically shuts down the cluster.
>>
>>> import subprocess
>>> import signal
>>> import time
>>
>>
>>> def signal_handler(signal,frame):
>>>      print("Shutting Down Cluster")
>>>      subprocess.Popen(["ipcluster", "stop", "--quiet"])
>>>      sys.exit(0)
>>
>>
>>>
>>> signal.signal(signal.SIGINT, signal_handler)
>>> subprocess.Popen(["ipcluster",
>>>                   "start",
>>>                   "--n=" + str(12),
>>>                   "--daemonize",
>>>                   "--quiet"])
>>> time.sleep(5) #time for the cluster to spin up
>>> #Code using cluster
>>> subprocess.Popen(["ipcluster", "stop", "--quiet"])
>>
>>
>> Hope this helps.
>>
>> Ben
>>
>> On Tue, Sep 25, 2012 at 1:13 PM, Brent Pedersen <bpederse@gmail.com>
>> wrote:
>>>
>>> Hi,
>>> I'm attempting to understand how I can use an ipengine on an LSF cluster.
>>> I've done something like this:
>>>
>>>      ipython profile create --parallel --profile=lsf
>>>      # edit .ipython/profile_lsf/ipcluster_config.py to use LSF
>>>      ipcluster start --profile=lsf --daemonize
>>>
>>> But, I'd like to be able to start the engines programatically.
>>> After the above command, I can do this:
>>>
>>>     c = Client(profile='lsf')
>>>
>>> Is there a way to "connect" to the profile and have it dynamically launch
>>> engines via python?
>>> I found the IPEngineApp, but it's not clear to me how to use that.
>>>
>>> Is there an example somewhere that launches engines via the API, rather
>>> than
>>> the command line? Even one for PBS would be helpful.
>>>
>>> thanks,
>>> -Brent
>>> _______________________________________________
>>> IPython-User mailing list
>>> IPython-User@scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-user
>>
>>
>>
>> _______________________________________________
>> IPython-User mailing list
>> IPython-User@scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-user
>>
>
>
> _______________________________________________
> IPython-User mailing list
> IPython-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-user
>