[Numpy-discussion] missing data discussion round 2

Mark Wiebe mwwiebe@gmail....
Thu Jun 30 10:11:10 CDT 2011

On Wed, Jun 29, 2011 at 4:21 PM, Eric Firing <efiring@hawaii.edu> wrote:

> On 06/29/2011 09:32 AM, Matthew Brett wrote:
> > Hi,
> >
> [...]
> >
> > Clearly there are some overlaps between what masked arrays are trying
> > to achieve and what Rs NA mechanisms are trying to achieve.  Are they
> > really similar enough that they should function using the same API?
> > And if so, won't that be confusing?  I think that's the question
> > that's being asked.
> And I think the answer is "no".  No more confusing to people coming from
> R to numpy than views already are--with or without the NEP--and not
> *requiring* people to use any NA-related functionality beyond what they
> are used to from R.
> My understanding of the NEP is that it directly yields an API closely
> matching that of R, but with the opportunity, via views, to do more with
> less work, if one so desires.  The present masked array module could be
> made more efficient if the NEP is implemented; regardless of whether
> this is done, the masked array module is not about to vanish, so anyone
> wanting precisely the masked array API will have it; and others remain
> free to ignore it (except for those of us involved in developing
> libraries such as matplotlib, which will have to support all variations
> of the new API along with the already-supported masked arrays).
> In addition, for new code, the full-blown masked array module may not be
> needed.  A convenience it adds, however, is the automatic masking of
> invalid values:
> In [1]: np.ma.log(-1)
> Out[1]: masked
> I'm sure this horrifies some, but there are times and places where it is
> a genuine convenience, and preferable to having to use a separate
> operation to replace nan or inf with NA or whatever it ends up being.

I added a mechanism to support this idea with the NA dtypes approach,
spelled 'NA[f8,InfNan]'. Here, all Infs and NaNs are treated as NA by the


If np.seterr were extended to allow such automatic masking as an option,
> then the need for a separate masked array module would shrink further.
> I wouldn't mind having to use an explicit kwarg for ignoring NA in
> reduction methods.
> Eric
> >
> > See you,
> >
> > Matthew
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion@scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20110630/2d0aefc8/attachment.html 

More information about the NumPy-Discussion mailing list