[Numpy-discussion] Masked arrays and pickle/unpickle

[Pierre GM]

On Thursday 17 July 2008 16:29:48 Stéfan van der Walt wrote:
> > An other possibility is to store the MaskedArray as a record array, with
> > one field for the data and one field for the mask.
>
> What about the other parameters, such as fill value? 

Dang, forgot about that. Having a dictionary of options would be cool, but we 
can't store it inside a regular ndarray. If we write to a file, we may want 
to write a header first that would store all the metadata we need.

> If we can come up with a robust way to convert a
> MaskedArray into (one or more) structured array(s), that would be
> perfect for storage purposes.  Also, you wouldn't need to be
> volunteered to implement it :)

A few weeks ago, I played a bit with interfacing TimeSeries and pytables: the 
idea is to transform the series (basically a MaskedArray) into a record 
array, and add the parameters such as fill_value in the metadata section of 
the table. Works great, we may want to follow the same pattern. Moreover, 
hdf5 is portable.

> Further, could we rename numpy.ma.core to numpy.ma._core?  I think we
> should make it clear that users should not import from core directly.

Anthony raised a very good point against that, and I agree. There's no need 
for that.


Anthony, just making a symlink from numpy/oldnumeric/ma.py to numpy/core/ma.py 
works to unpickle your array. I agree it's still impractical...