[Numpy-discussion] feedback request: proposal to add masks to the core ndarray

Mark Wiebe mwwiebe@gmail....
Fri Jun 24 13:14:12 CDT 2011


On Fri, Jun 24, 2011 at 11:25 AM, Christopher Barker
<Chris.Barker@noaa.gov>wrote:

> Robert Kern wrote:
>
> > It's worth noting that this is not a replacement for masked arrays,
> > nor is it intended to be the be-all, end-all solution to missing data
> > problems. It's mostly just intended to be a focused tool to fill in
> > the gaps where masked arrays are less convenient for whatever reason;
>
> I think we have enough problems with ndarray vs numpy.ma -- this is
> introducing a third option? IMHO, and from the discussion, it seems this
> proposal should be a "uniter, not a divider".
>
> While masked arrays may still exist, it would be nice if they were an
> extension of the new built-in thingie, not yet another implementation.
>

If someone wants to have the current numpy.ma semantics with regards to
NaN's becoming masked automatically, or some other behavior not in my
proposal, I think a subclass with tweaks would be good, yes.

>  Not every dtype
> > would have an NA-aware counterpart.
>
> One of the great things about numpy is the full range of data types. I
> think it would be surprising and frustrating not to have masked versions
> of them all.
>
> By the way, what might be the performance hit of a "new" dtype --
> wouldn't we lose all sort of opportunities for the compiler and hardware
> to optimize? I can only image an "if" statement with every single
> computation. But maybe that isn't any more of a hit that a separate mask.
>

There is also a potential reliability issue with the na* dtype approach.
Every single operation of every dtype has to get the NA logic correct. While
there may be ways to unify the code in various ways, ensuring robustness of
that is more difficult than for a system which has a single implementation
of the NA logic applied to masks.

-Mark


>
> -Chris
>
>
> --
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker@noaa.gov
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20110624/ec6d6464/attachment-0001.html 


More information about the NumPy-Discussion mailing list