[Numpy-discussion] alterNEP - was: missing data discussion round 2

Lluís xscript@gmx....
Thu Jun 30 14:06:21 CDT 2011

Nathaniel Smith writes:

> On Thu, Jun 30, 2011 at 11:27 AM, Lluís <xscript@gmx.net> wrote:
>> As I tried to convey on my other mail, separating both will force you to
>> either:
>> * Make a copy of the array before passing it to another routine (because
>>  the routine will assign np.NA but you still want the original data)

> To help me understand, do you have an example in mind of a routine
> that would do that? I can't think of any cases where I had some
> original data that some routine wanted to throw out and replace with
> NAs; it just seems... weird. Maybe I'm missing something though...

Well, I had some silly example on another thread. A function that
computes the mean of all non-NA values, and assigns NA to all cells that
are beyond certain threshold of that mean value.

> (I can imagine that it would make sense for what we're calling a
> masked array, where you have some routine which computes which values
> should be ignored for a particular purpose. But if it only makes sense
> for masked arrays then you can just write your routine to work with
> masked arrays only, and it doesn't matter how similar the masking and
> missing APIs are.)

The routine makes sense by itself as a beyond-mean detector.

The routine must not care whether your NAs are transient or not (in your
aNEP, whether you want it to assign np.NA or np.IGNORE, which must be
indicated by the caller through yet another function argument).

Note that callers will not only have to indicate which "type" of missing
data the calle should use (np.NA or np.IGNORE), but they also have to
indicate whether np.NAs must be ignored (i.e., skipna=bool), as well as
np.IGNORE (i.e., propmask=bool).

Of course it is doable, but adding 3 more arguments to *all* functions
(including ufuncs, and higher-level functions) does not seem as
desirable to me.


 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom

More information about the NumPy-Discussion mailing list