[Numpy-discussion] Medians that ignore values

[Peter Saffrey]

  <jh <at> physics.ucf.edu> writes:

 > Currently the only way you can handle NaNs is by using masked arrays.
 > Create a mask by doing isfinite(a), then call the masked array
 > median().  There's an example here:
 >
 > http://sd-2116.dedibox.fr/pydocweb/doc/numpy.ma/
 >

I had looked at masked arrays, but couldn't quite get them to work. 
Generating them is fine (I've randomly introduced a few nan values into 
this array):

 >>> from numeric import *
 >>> from pylab import rand
 >>> a = rand(10,3)
 >>> a[a > 0.8] = nan
 >>> m = ma.masked_array(a, isnan(a))
 >>> m
array(data =
  [[  5.97400164e-01   1.00000000e+20   1.00000000e+20]
  [  3.34623242e-01   6.53582662e-02   2.12298948e-01]
  [  2.11879853e-01   1.00000000e+20   3.57822574e-01]
  [  6.06911592e-01   1.96229341e-01   5.49953059e-02]
  [  1.00000000e+20   2.75493584e-01   4.70929957e-01]
  [  2.92845118e-01   2.11261529e-02   3.49211381e-02]
  [  7.11963636e-01   2.17277855e-01   5.45487384e-02]
  [  5.20995579e-01   7.57676845e-01   1.00000000e+20]
  [  1.84189196e-01   7.58291436e-02   6.26567116e-01]
  [  2.42083978e-01   1.00000000e+20   2.30202562e-02]],
       mask =
  [[False  True  True]
  [False False False]
  [False  True False]
  [False False False]
  [ True False False]
  [False False False]
  [False False False]
  [False False  True]
  [False False False]
  [False  True False]],
       fill_value=1e+20)


Remember I want medians of each triple, so I need to median the 
transposed matrix:

 >>> median(m.T)
array([  1.00000000e+20,   2.12298948e-01,   3.57822574e-01,
          1.96229341e-01,   4.70929957e-01,   3.49211381e-02,
          2.17277855e-01,   7.57676845e-01,   1.84189196e-01,
          2.42083978e-01])

The first value is NaN, indicating that the median routine has failed to 
ignore the masked values. What have I missed?

Thanks,

Peter