[Numpy-discussion] Warnings in numpy.ma.test()
Wed Mar 17 07:19:45 CDT 2010
On Wed, Mar 17, 2010 at 2:07 AM, Pierre GM <email@example.com> wrote:
> As you're probably aware, the current test suite for numpy.ma raises some nagging warnings such as "invalid value in ...". These warnings are only issued when a standard numpy ufunc (eg., np.sqrt) is called on a MaskedArray, instead of its numpy.ma (eg., np.ma.sqrt) equivalent. The reason is that the masked versions of the ufuncs temporarily set the numpy error status to 'ignore' before the operation takes place, and reset the status to its original value.
> I thought I could use the new __array_prepare__ method to intercept the call of a standard ufunc. After actual testing, that can't work. __array_prepare only help to prepare the *output* of the operation, not to change the input on the fly, just for this operation. Actually, you can modify the input in place, but it's usually not what you want.
That is correct, __array_prepare__ is called just after the output
array is created, but before the ufunc actually gets down to business.
I have the same limitation in quantities you are now seeing with
masked array, in my case I want the opportunity to rescale different
but compatible quantities for the operation (without changing the
original arrays in place, of course).
> Then, I tried to use __array_prepare__ to store the current error status in the input, force it to ignore divide/invalid errors and send the input to the ufunc. Doesn't work either: np.seterr in __array_prepare__ does change the error status, but as far as I understand, the ufunc is called is still called with the original error status. That means that if something goes wrong, your error status can stay stuck. Not a good idea either.
> I'm running out of ideas at this point. For the test suite, I'd suggest to disable the warnings in test_fix_invalid and test_basic_arithmetic.
> An additional issue is that if one of the error status is set to 'raise', the numpy ufunc will raise the exception (as expected), while its numpy.ma version will not. I'll put also a warning in the docs to that effect.
> Please send me your comments before I commit any changes.
I started thinking about a third method called __input_prepare__ that
would be called on the way into the ufunc, which would allow you to
intercept the input and pass a somehow modified copy back to the
ufunc. The total flow would be:
1) Call myufunc(x, y[, z])
2) myufunc calls ?.__input_prepare__(myufunc, x, y), which returns x',
y' (or simply passes through x,y by default)
3) myufunc creates the output array z (if not specified) and calls
?.__array_prepare__(z, (myufunc, x, y, ...))
4) myufunc finally gets around to performing the calculation
5) myufunc calls ?.__array_wrap__(z, (myufunc, x, y, ...)) and returns
the result to the caller
Is this general enough for your use case? I haven't tried to think
about how to change some global state at one point and change it back
at another, that seems like a bad idea and difficult to support.
More information about the NumPy-Discussion