[Numpy-discussion] feedback request: proposal to add masks to the core ndarray
Tue Jun 28 07:42:59 CDT 2011
Charles R Harris writes:
> I think we may need some standard format for masked data on disk if we
> don't go the NA value route.
As I see it, the mask array is just some metadata that is attached to
the dtype descriptor. I don't know how an ndarray is (un)pickled from
disk, but I imagine that each dtype descriptor can be
(de)serialized. Thus this will also include the mask array. Note that if
the mask array is metadata attached to the dtype, structured arrays (or
however they're called nowadays) can have different mask arrays for each
of the struct fields, or share them arbitrarily. In any case, pickling
will take care of storing just once each of the mask arrays.
This reminds me that I've wanted for long time to store extra metadata
on the dtypes. What I've found out is that metadata is stored on the
dtype that describes the structure of the whole array, not on the
per-field dtype. This makes it harder to retain per-field metadata
whenever you operate on a field-per-field basis.
"And it's much the same thing with knowledge, for whenever you learn
something new, the whole world becomes that much richer."
-- The Princess of Pure Reason, as told by Norton Juster in The Phantom
More information about the NumPy-Discussion