[Numpy-discussion] Handling interrupts in NumPy extensions
david at ar.media.kyoto-u.ac.jp
Fri Aug 25 05:12:57 CDT 2006
Travis Oliphant wrote:
> David Cournapeau wrote:
>> By the way, I tried something for python.thread + signals. This is posix
>> specific, and it works as expected on linux:
> Am I right that this could this be accomplished simply by throwing away
> all the interrupt handling stuff in the code and checking for
> PyOS_InterruptOccurred() in the place where you check for the global
> variable that your signal handler uses? Your signal handler does
> essentially what Python's signal handler already does, if I'm not mistaken.
I don't know how the python signal handler works, but I believe it should do
more or less the same, indeed. The key idea is that it is important to mask
other signals related to interrupting. To have a relatively clear view
on this, if you have not seen it, you may take a look at the gnu C doc
on signal handling:
After having given some thought, I am wondering about what exactly we
are trying to do:
- the main problem is to be able to interrupt some which may take a
long time to compute, without corrupting the whole python process.
- for that, those function need to be able to trap the usual signals
corresponding to interrupt (SIGINT, etc... on Unix, equivalents on windows).
There are two ways to handle a signal:
- check regularly some global (that is, global to the whole process)
value, and if change this value if a signal is trapped. That's the
easier way, but this is not thread safe as I first thought (I will code
an example if I have time).
- the signal handler jumps to an other point of the program where
cleaning is done: this is more complicated, and I am not sure we need
the complication (I have never used this scheme, so I may just miss the
point totally). I don't even want to think how it works in
multi-threading environment :)
Now, the threading issue came in, and I am not sure why we need to care:
this is a problem if numpy is implemented in a multi-thread way, but I
don't believe it to be the case, right ? An other solution, which is
used I think in more sophisticated programs, is having one thread with
high priority, which only job is to detect signals, and to mask all
signals in all other threads. Again, this seems overkill (and highly
non portable) ? And this should be the python interpreter job, no ?
Actually, as this is a generic problem for any python extension code,
other really smart people should have thought about that... If I am
interpreting correctly what is said here
http://docs.python.org/lib/module-signal.html, I believe that what you
suggest (using PyOS_InterruptOccurred() at some points) is what shall be
done: the python interpreter is making sure that the signal is send to
the main thread, that is the thread where numpy is executed (that's my
understanding on the way python interpreter works, not a fact).
More information about the Numpy-discussion