[Numpy-discussion] numpy error handling

[Travis Oliphant]

Tim Hochberg wrote:
>
> I've just been looking at how numpy handles changing the behaviour 
> that is triggered when there are numeric error conditions (overflow, 
> underflow, etc.). If I understand it correctly, and that's a big if, I 
> don't think I like it nearly as much as the what numarray has in place.
>
> It appears that numpy uses the two functions, seterr and geterr, to 
> set and query the error handling. These set/read a secret variable 
> stored in the local scope. 
This approach was decided on after discussions with Guido who didn't 
like the idea of pushing and popping from a global stack.    I'm not 
sure I'm completely in love with it my self, but it is actually more 
flexible then the numarray approach.

You can get the numarray approach back simply by setting the error in 
the builtin scope (instead of in the local scope which is done by default. 

Then, at the end of the function, you can restore it.  If it was felt 
useful to create a stack to handle this on the builtin level then that 
is easily done as well.

> I assume that the various ufuncs then examine that value to determine 
> how to handle errors. The secret variable approach is a little clunky, 
> but that's not what concerns me. What concerns me is that this 
> approach is *only* useful for built in numpy functions and falls down 
> if we call any user defined functions.
>
> Suppose we want to be warned on underflow. Setting this is as simple as:
>
>    def func(*args):
>        numpy.seterr(under='warn')
>        # do stuff with args
>        return result
>
> Since seterr is local to the function, we don't have to reset the 
> error handling at the end, which is convenient. And, this works fine 
> if all we are doing is calling numpy functions and methods. However, 
> if we are calling a function of our own devising we're out of luck 
> since the called function will not inherit the error settings that we 
> have set.
Again, you have control over where you set the "secret" variable (local, 
global (module), and builtin).  I also don't see how that's anymore 
clunky then a "secret" stack.   You may set the error in the builtin 
scope --- in fact it would probably be trivial to implement a stack 
based on this and implement the

pushMode
popMode

interface of numarray.


But, I think this question does deserve a bit of debate.  I don't think 
there has been a serious discussion over the method.  To help Tim and 
others understand what happens:

When a ufunc is called, a specific variable name is searched for in the 
following name-spaces in the following order:

1) local
2) global
3) builtin

(There is a bit of an optimization in that when the error mode is the 
default mode --- do nothing, a global flag is set which by-passes the 
search for the name). 

The first time the variable name is found, the error mode is read from 
that variable.  This error mode is placed as part of the ufunc loop 
object.  At the end of each 1-d loop the IEEE error mode flags are 
checked  (depending on the state of the error mode) and appropriate 
action taken.

By the way, it would not be too difficult to change how the error mode 
is set (probably an hour's worth of work).   So, concern over 
implementation changes should not be a factor right now.   

Currently the error mode is read from a variable using standard scoping 
rules.   It would save the (not insignificant) name-space lookup time to 
instead use a global stack (i.e. a Python list) and just get the error 
mode from the top of that stack.

> Thus we have no way to influence the error settings of functions 
> downstream from us.
Of course, there is a way to do this by setting the variable in the 
global or builtin scope as I've described above. 

What's really the argument here, is whether having the flexibility at 
the local and global name-spaces really worth the extra name-lookups for 
each ufunc.

I've argued that the numarray behavior can result from using the builtin 
namespace for the error control. (perhaps with better Python-side 
support for setting and retrieving it).  What numpy has is control at 
the global and local namespace level as well which can override the 
builtin name-space behavior.

So, we should at least frame the discussion in terms of what is actually 
possible. 
>
> I also would prefer more verbose keys ala numarray (underflow, 
> overflow, dicidebyzero and invalid) than those currently used by numpy 
> (under, over, divide and invalid). 

In my mind, verbose keys are just extra baggage unless they are really 
self documenting.  You just need reminders and clues.   It seems to be a 
preference thing.   I guess I hate typing long strings when only the 
first few letters clue me in to what is being talked about. 

> And (will he never stop) I like numarrays defaults better here too: 
> overflow='warn', underflow='ignore', dividebyzero='warn', 
> invalid='warn'. Currently, numpy defaults to ignore for all cases. 
> These last points are relatively minor though.
This has optimization issues the way the code is written now.  The 
defaults are there to produce the fastest loops.   So, I'm hesitant to 
change them based only on ambiguous preferences.

Good feedback.    Thanks again for taking the time to look at this and 
offer review.

-Travis








>
> Regards,
>
> -tim
>
>
> [1] I believe it's actually thread local, and if it's not it should 
> be, but that detail is not important for our discussion here.
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting 
> language
> that extends applications into web and mobile media. Attend the live 
> webcast
> and join the prime developer group breaking into this new coding 
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion