A reimplementation of MaskedArray

A. M. Archibald peridot.faceted at gmail.com
Wed Nov 8 12:41:55 CST 2006


On 08/11/06, Pierre GM <pgmdevlist at gmail.com> wrote:

> I like your idea, but not its implementation. If MA.masked_singleton is
> defined as an object, as you suggest, then the dtype of the ndarray it is
> passed to becomes 'object', as you pointed out, and that is not something one
> would naturally expec, as basic numerical functions don't  work well  with the
> 'object' dtype (just try  N.sqrt(N.array([1],dtype=N.object)) to see what I
> mean).
> Even if we can construct a mask rather easily at the creation of the masked
> array, following your 'a==masked' suggestion, we still need to get the dtype
> of the non-masked section, and that doesn't seem trivial...

A good candidate for "should be masked" marked is NaN. It is supposed
to mean, more or less, "no sensible value". Unfortunately, integer
types do not have such a special value. It's also conceivable that
some user might want to keep NaNs in their array separate from the
mask. Finally, on some hardware, operations with NaN are very slow (so
leaving them in the array, even masked, might not be a good idea).

The reason I suggest this is that in the last major application I had
for numpy, one stage of the problem would occasionally result in NaNs
for certain values, but the best thing I could do was leave them in
place to represent "no data". Switching to a MaskedArray might have
been a better idea, but the NaNs were a rare occurrence.

> About the conversion to ndarray:
> By default, the result should have the same dtype as the _data section.
> For this reason, I disagree with your idea of "(returning) an object ndarray
> with the missing value containing the masked singleton". If you really want
> an object ndarray, you can use the filled method or the filled function, with
> your own definition of the filling value (such as your MaskedScalar).

If you've got floating point, you can again fill in NaNs, but you have
a good point about wanting to extract the original values that were
masked out. Depending on what one is doing, one might want one or the
other.

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642




More information about the Numpy-discussion mailing list