[Numpy-discussion] numpy error handling
oliphant.travis at ieee.org
Fri Mar 31 23:44:01 CST 2006
Tim Hochberg wrote:
> I've just been looking at how numpy handles changing the behaviour
> that is triggered when there are numeric error conditions (overflow,
> underflow, etc.). If I understand it correctly, and that's a big if, I
> don't think I like it nearly as much as the what numarray has in place.
> It appears that numpy uses the two functions, seterr and geterr, to
> set and query the error handling. These set/read a secret variable
> stored in the local scope.
This approach was decided on after discussions with Guido who didn't
like the idea of pushing and popping from a global stack. I'm not
sure I'm completely in love with it my self, but it is actually more
flexible then the numarray approach.
You can get the numarray approach back simply by setting the error in
the builtin scope (instead of in the local scope which is done by default.
Then, at the end of the function, you can restore it. If it was felt
useful to create a stack to handle this on the builtin level then that
is easily done as well.
> I assume that the various ufuncs then examine that value to determine
> how to handle errors. The secret variable approach is a little clunky,
> but that's not what concerns me. What concerns me is that this
> approach is *only* useful for built in numpy functions and falls down
> if we call any user defined functions.
> Suppose we want to be warned on underflow. Setting this is as simple as:
> def func(*args):
> # do stuff with args
> return result
> Since seterr is local to the function, we don't have to reset the
> error handling at the end, which is convenient. And, this works fine
> if all we are doing is calling numpy functions and methods. However,
> if we are calling a function of our own devising we're out of luck
> since the called function will not inherit the error settings that we
> have set.
Again, you have control over where you set the "secret" variable (local,
global (module), and builtin). I also don't see how that's anymore
clunky then a "secret" stack. You may set the error in the builtin
scope --- in fact it would probably be trivial to implement a stack
based on this and implement the
interface of numarray.
But, I think this question does deserve a bit of debate. I don't think
there has been a serious discussion over the method. To help Tim and
others understand what happens:
When a ufunc is called, a specific variable name is searched for in the
following name-spaces in the following order:
(There is a bit of an optimization in that when the error mode is the
default mode --- do nothing, a global flag is set which by-passes the
search for the name).
The first time the variable name is found, the error mode is read from
that variable. This error mode is placed as part of the ufunc loop
object. At the end of each 1-d loop the IEEE error mode flags are
checked (depending on the state of the error mode) and appropriate
By the way, it would not be too difficult to change how the error mode
is set (probably an hour's worth of work). So, concern over
implementation changes should not be a factor right now.
Currently the error mode is read from a variable using standard scoping
rules. It would save the (not insignificant) name-space lookup time to
instead use a global stack (i.e. a Python list) and just get the error
mode from the top of that stack.
> Thus we have no way to influence the error settings of functions
> downstream from us.
Of course, there is a way to do this by setting the variable in the
global or builtin scope as I've described above.
What's really the argument here, is whether having the flexibility at
the local and global name-spaces really worth the extra name-lookups for
I've argued that the numarray behavior can result from using the builtin
namespace for the error control. (perhaps with better Python-side
support for setting and retrieving it). What numpy has is control at
the global and local namespace level as well which can override the
builtin name-space behavior.
So, we should at least frame the discussion in terms of what is actually
> I also would prefer more verbose keys ala numarray (underflow,
> overflow, dicidebyzero and invalid) than those currently used by numpy
> (under, over, divide and invalid).
In my mind, verbose keys are just extra baggage unless they are really
self documenting. You just need reminders and clues. It seems to be a
preference thing. I guess I hate typing long strings when only the
first few letters clue me in to what is being talked about.
> And (will he never stop) I like numarrays defaults better here too:
> overflow='warn', underflow='ignore', dividebyzero='warn',
> invalid='warn'. Currently, numpy defaults to ignore for all cases.
> These last points are relatively minor though.
This has optimization issues the way the code is written now. The
defaults are there to produce the fastest loops. So, I'm hesitant to
change them based only on ambiguous preferences.
Good feedback. Thanks again for taking the time to look at this and
>  I believe it's actually thread local, and if it's not it should
> be, but that detail is not important for our discussion here.
> This SF.Net email is sponsored by xPML, a groundbreaking scripting
> that extends applications into web and mobile media. Attend the live
> and join the prime developer group breaking into this new coding
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
More information about the Numpy-discussion