[Numpy-discussion] masked arrays and record arrays

[John Hunter]

On 6/12/07, Robert Kern <robert.kern@gmail.com> wrote:
> John Hunter wrote:
> > Do record arrays support masks?
>
> I believe so, but not individual masks for each component in the record.

I see, too bad.  I am working on the matplotlib.mlab.csv2rec function
and need to handle missing values properly.  For floats, I can use
numpy.nan, but for other types, eg int, it would be nice to be able to
set a column dependent mask.  I did manage to create a mask on an
entire row of a record array, and it is working OK, but there appears
to be a bug in repr

In [191]: from matplotlib.mlab import csv2rec

In [192]: import numpy

In [193]: import numpy.core.ma as ma

In [194]: r = csv2rec('data/msft.csv')

In [195]: mask = numpy.zeros(len(r), numpy.bool_)

In [196]: mask[0] = 1

In [197]: m = ma.masked_where(mask, r)

In [198]: m[0]
Out[198]:
array(data =
 999999,
      mask =
 True,
      fill_value=999999)


In [199]: m[1]
Out[199]: (datetime.datetime(2003, 9, 18, 0, 0), 28.489999999999998,
29.510000000000002, 28.420000000000002, 29.5, 67268096, 29.34)

In [200]: m
Out[200]: ------------------------------------------------------------
Traceback (most recent call last):
  File "<ipython console>", line 1, in ?
  File "/home/titan/johnh/dev/lib/python2.4/site-packages/IPython/Prompts.py",
line 517, in __call__
    manipulated_val = self.display(arg)
  File "/home/titan/johnh/dev/lib/python2.4/site-packages/IPython/Prompts.py",
line 543, in _display
    return self.shell.hooks.result_display(arg)
  File "/home/titan/johnh/dev/lib/python2.4/site-packages/IPython/hooks.py",
line 134, in __call__
    ret = cmd(*args, **kw)
  File "/home/titan/johnh/dev/lib/python2.4/site-packages/IPython/hooks.py",
line 162, in result_display
    out = pformat(arg)
  File "/opt/app/g++lib6/python-2.4/lib/python2.4/pprint.py", line
110, in pformat
    self._format(object, sio, 0, 0, {}, 0)
  File "/opt/app/g++lib6/python-2.4/lib/python2.4/pprint.py", line
128, in _format
    rep = self._repr(object, context, level - 1)
  File "/opt/app/g++lib6/python-2.4/lib/python2.4/pprint.py", line 194, in _repr
    self._depth, level)
  File "/opt/app/g++lib6/python-2.4/lib/python2.4/pprint.py", line
206, in format
    return _safe_repr(object, context, maxlevels, level)
  File "/opt/app/g++lib6/python-2.4/lib/python2.4/pprint.py", line
291, in _safe_repr
    rep = repr(object)
  File "/opt/tradelink/research/site-packages/numpy/core/ma.py", line
760, in __repr__
    return with_mask % {
  File "/opt/tradelink/research/site-packages/numpy/core/ma.py", line
1233, in filled
    result[m] = value
ValueError: tried to set void-array with object members using buffer.