[Numpy-discussion] Handling interrupts in NumPy extensions

Perry Greenfield perry at stsci.edu
Wed Aug 23 16:43:15 CDT 2006


I thought it might be useful to give a little more context on the  
problems involved in handling such interruptions. Basically, one  
doesn't want to exit out of places where data structures are  
incompletely set up, or memory isn't properly handled so that later  
references to these don't cause segfaults (or experience memory leaks).

There may be more exotic cases but typically many extensions are as  
simple as:

1) Figure out what inputs one has and the mode of computation needed
2) allocate and setup output arrays
3) do computation, possibly lengthy, over arrays
4) free temporary arrays and other data structures
5) return results

Typically, the interrupt handling is needed only for 3, the part that  
it may spend a very long time in. 1, 2, 4, and 5 are not worth  
interrupting, and the area that may cause the most trouble. I'd argue  
that many things could do with a very simple structure where section  
3 is bracketed with macros. Something like:

NPY_SIG_INTERRUPTABLE
[long looping computational code that doesn't create or destroy objects]
NPY_SIG_END_INTERRUPTABLE

followed by the normal code to do 4 and 5. What happens during an  
interrupt is the computation code is exited and execution resumes  
right after the closing macro. Very often  one doesn't care that the  
results in the arrays may be incomplete, or invalid numbers  
(presumably you know that since you just did control-C, but maybe I'm  
confused). Any reason that most cases couldn't be handled with  
something this simple?

All cases can't be handled with this, but most should I think.

Perry

On Aug 23, 2006, at 2:45 PM, Travis Oliphant wrote:

>
> I'm working on some macros that will allow extensions to be
> "interruptable" (i.e. with Ctrl-C).  The idea came from SAGE but the
> implementation is complicated by the possibility of threads and making
> sure to handle clean-up code correctly when the interrupt returns.
>
> I'd like to get this in to 1.0 final.  Anything needed will not  
> require
> re-compilation of extension modules built for 1.0b2 however.  This  
> will
> be strictly "extra" and if an extension module doesn't use it there  
> will
> be no problems.
>
> Step 1:
>
> Define the interface.  Here are a couple of draft proposals.  Please
> comment on them.
>
> 1) General purpose interface
>
> NPY_SIG_TRY {
> [code]
> }
> NPY_SIG_EXCEPT(signum) {
> [interrupt handling return]
> }
> NPY_SIG_ELSE
> [normal return]
>
> The idea of signum is to hold the signal actually caught.
>
>
> 2) Simpler interface
>
> NPY_SIG_TRY {
> [code]
> }
> NPY_SIG_EXCEPT_GOTO(label)
> [normal return]
>
> label:
>   [interrupt handling return]
>
>
> C-extensions often use the notion of a label to handle failure code.
>
> If anybody has any thoughts on this, they would be greatly  
> appreciated.
>
>
> Step 2:
>
> Implementation.  I have the idea to have a single interrupt handler
> (defined globally in NumPy) that basically uses longjmp to return  
> to the
> section of code corresponding to the thread that is handling the
> interrupt.  I had thought to use a global variable containing a linked
> list of jmp_buf structures with a thread-id attached
> (PyThread_get_thread_ident()) so that the interrupt handler can search
> it to see if the thread has registered a return location.  If it has
> not, then the intterupt handler will just return normally.   In  
> this way
> a thread that calls setjmpbuf will be sure to return to the correct
> place when it handles the interrupt.
>
> Concern:
>
> My thinking is that this mechanism should work whether or not the  
> GIL is
> held so that we don't have to worry about whether or not the GIL is  
> held
> except in the interrupt handling case (when Python exceptions are  
> to be
> set).    But, honestly, this gets very confusing.
>
> The sigjmp / longjmp mechanism for handling interrupts is not
> recommended under windows (not sure about mingw), but there we could
> possibly use Microsoft's __try and __except extension to implement.
> Initially, it would be "un-implemented" on platforms where it  
> didn't work.
>
> Any comments are greatly appreciated
>
> -Travis
>
>
>
>
> ---------------------------------------------------------------------- 
> ---
> Using Tomcat but need to do more? Need to support web services,  
> security?
> Get stuff done quickly with pre-integrated technology to make your  
> job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache  
> Geronimo
> http://sel.as-us.falkag.net/sel? 
> cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion





More information about the Numpy-discussion mailing list