[Numpy-svn] r6320 - in trunk: doc/source/reference numpy/ma

numpy-svn@scip... numpy-svn@scip...
Tue Jan 13 15:02:03 CST 2009


Author: pierregm
Date: 2009-01-13 15:01:58 -0600 (Tue, 13 Jan 2009)
New Revision: 6320

Added:
   trunk/doc/source/reference/maskedarray.baseclass.rst
   trunk/doc/source/reference/maskedarray.generic.rst
   trunk/doc/source/reference/maskedarray.rst
Modified:
   trunk/doc/source/reference/arrays.classes.rst
   trunk/doc/source/reference/arrays.ndarray.rst
   trunk/numpy/ma/core.py
   trunk/numpy/ma/extras.py
Log:
numpy.ma.core:
* introduced baseclass, sharedmask and hardmask as readonly properties of MaskedArray
* docstrings update

numpy.ma.extras:
* docstring updates

docs/reference
* introduced maskedarray, maskedarray.baseclass, maskedarray.generic

Modified: trunk/doc/source/reference/arrays.classes.rst
===================================================================
--- trunk/doc/source/reference/arrays.classes.rst	2009-01-12 21:25:53 UTC (rev 6319)
+++ trunk/doc/source/reference/arrays.classes.rst	2009-01-13 21:01:58 UTC (rev 6320)
@@ -261,8 +261,13 @@
 Masked arrays (:mod:`numpy.ma`)
 ===============================
 
-.. seealso:: :ref:`routines.ma`
+.. seealso:: :ref:`maskedarray`
 
+Masked arrays are arrays that may have missing or invalid entries.
+The :mod:`numpy.ma` module provides a nearly work-alike replacement for numpy
+that supports data arrays with masks.
+
+
 .. XXX: masked array documentation should be improved
 
 .. currentmodule:: numpy
@@ -270,14 +275,12 @@
 .. index::
    single: masked arrays
 
-.. autosummary::
-   :toctree: generated/
+.. toctree::
+   :maxdepth: 2
 
-   ma.masked_array
+   maskedarray
 
-.. automodule:: numpy.ma
 
-
 Standard container class
 ========================
 

Modified: trunk/doc/source/reference/arrays.ndarray.rst
===================================================================
--- trunk/doc/source/reference/arrays.ndarray.rst	2009-01-12 21:25:53 UTC (rev 6319)
+++ trunk/doc/source/reference/arrays.ndarray.rst	2009-01-13 21:01:58 UTC (rev 6320)
@@ -135,6 +135,8 @@
    is automatically made.
 
 
+.. _arrays.ndarray.attributes:
+
 Array attributes
 ================
 
@@ -217,6 +219,9 @@
 
 .. note:: XXX: update and check these docstrings.
 
+
+.. _array.ndarray.methods:
+
 Array methods
 =============
 

Added: trunk/doc/source/reference/maskedarray.baseclass.rst
===================================================================
--- trunk/doc/source/reference/maskedarray.baseclass.rst	2009-01-12 21:25:53 UTC (rev 6319)
+++ trunk/doc/source/reference/maskedarray.baseclass.rst	2009-01-13 21:01:58 UTC (rev 6320)
@@ -0,0 +1,395 @@
+
+.. currentmodule:: numpy.ma
+
+
+.. _numpy.ma.constants:
+
+Constants of the :mod:`numpy.ma` module
+=======================================
+
+In addition to the :class:`MaskedArray` class, the :mod:`numpy.ma` module
+defines several constants.
+
+.. data:: masked
+
+   The :attr:`masked` constant is a special case of :class:`MaskedArray`,
+   with a float datatype and a null shape. It is used to test whether a
+   specific entry of a masked array is masked, or to mask one or several
+   entries of a masked array::
+
+      >>> x = ma.array([1, 2, 3], mask=[0, 1, 0])
+      >>> x[1] is ma.masked
+      True
+      >>> x[-1] = ma.masked
+      >>> x
+      masked_array(data = [1 -- --],
+                   mask = [False  True  True],
+             fill_value = 999999)
+
+
+.. data:: nomask
+
+   Value indicating that a masked array has no invalid entry.
+   :attr:`nomask` is used internally to speed up computations when the mask
+   is not needed.
+
+
+.. data:: masked_print_options
+
+   String used in lieu of missing data when a masked array is printed.
+   By default, this string is ``'--'``.
+
+
+
+
+.. _maskedarray.baseclass:
+
+The :class:`MaskedArray` class
+==============================
+
+ An instance of :class:`MaskedArray` can be thought as the combination of several elements:
+
+* The :attr:`data`, as a regular :class:`numpy.ndarray` of any shape or datatype (the data).
+* A boolean :attr:`mask` with the same shape as the data, where a ``True`` value indicates that the corresponding element of the data is invalid.
+  The special value :attr:`nomask` is also acceptable for arrays without named fields, and indicates that no data is invalid.
+* A :attr:`fill_value`, a value that may be used to replace the invalid entries in order to return a standard :class:`numpy.ndarray`.
+
+
+
+Attributes and properties of masked arrays
+------------------------------------------
+
+.. seealso:: :ref:`Array Attributes <arrays.ndarray.attributes>`
+
+
+.. attribute:: MaskedArray.data
+
+   Returns the underlying data, as a view of the masked array.
+   If the underlying data is a subclass of :class:`numpy.ndarray`, it is
+   returned as such.
+
+      >>> x = ma.array(np.matrix([[1, 2], [3, 4]]), mask=[[0, 1], [1, 0]])
+      >>> x.data
+      matrix([[1, 2],
+              [3, 4]])
+
+   The type of the data can be accessed through the :attr:`baseclass`
+   attribute.
+
+.. attribute:: MaskedArray.mask
+
+   Returns the underlying mask, as an array with the same shape and structure
+   as the data, but where all fields are booleans. 
+   A value of ``True`` indicates an invalid entry.
+
+
+.. attribute:: MaskedArray.recordmask
+
+   Returns the mask of the array if it has no named fields. For structured
+   arrays, returns a ndarray of booleans where entries are ``True`` if **all**
+   the fields are masked, ``False`` otherwise::
+
+      >>> x = ma.array([(1, 1), (2, 2), (3, 3), (4, 4), (5, 5)],
+      ...         mask=[(0, 0), (1, 0), (1, 1), (0, 1), (0, 0)],
+      ...        dtype=[('a', int), ('b', int)])
+      >>> x.recordmask
+      array([False, False,  True, False, False], dtype=bool)
+
+
+.. attribute:: MaskedArray.fill_value
+
+   Returns the value used to fill the invalid entries of a masked array.
+   The value is either a scalar (if the masked array has no named fields),
+   or a 0d-ndarray with the same datatype as the masked array if it has
+   named fields.
+
+   The default filling value depends on the datatype of the array:
+
+   ========  ========
+   datatype  default
+   ========  ========
+   bool      True
+   int       999999
+   float     1.e20
+   complex   1.e20+0j
+   object    '?'
+   string    'N/A'
+   ========  ========
+   
+
+
+.. attribute:: MaskedArray.baseclass
+
+   Returns the class of the underlying data.::
+
+      >>> x =  ma.array(np.matrix([[1, 2], [3, 4]]), mask=[[0, 0], [1, 0]])
+      >>> x.baseclass
+      <class 'numpy.core.defmatrix.matrix'>
+
+
+.. attribute:: MaskedArray.sharedmask
+
+   Returns whether the mask of the array is shared between several arrays.
+   If this is the case, any modification to the mask of one array will be
+   propagated to the other masked arrays.
+
+
+.. attribute:: MaskedArray.hardmask
+
+   Returns whether the mask is hard (``True``) or soft (``False``).
+   When the mask is hard, masked entries cannot be unmasked.
+
+
+As :class:`MaskedArray` is a subclass of :class:`~numpy.ndarray`, a masked array also inherits all the attributes and properties of a  :class:`~numpy.ndarray` instance.
+
+.. autosummary::
+   :toctree: generated/
+
+   MaskedArray.flags
+   MaskedArray.shape
+   MaskedArray.strides
+   MaskedArray.ndim
+   MaskedArray.size
+   MaskedArray.itemsize
+   MaskedArray.nbytes
+   MaskedArray.base
+   MaskedArray.dtype
+   MaskedArray.T
+   MaskedArray.real
+   MaskedArray.imag
+   MaskedArray.flat
+   MaskedArray.ctypes
+   MaskedArray.__array_priority__
+
+
+
+:class:`MaskedArray` methods
+============================
+
+.. seealso:: :ref:`Array methods <array.ndarray.methods>`
+
+
+Conversion
+----------
+
+.. autosummary::
+   :toctree: generated/
+
+   MaskedArray.view
+   MaskedArray.astype
+   MaskedArray.filled
+   MaskedArray.tofile
+   MaskedArray.toflex
+   MaskedArray.tolist
+   MaskedArray.torecords
+   MaskedArray.tostring
+
+
+Shape manipulation
+------------------
+
+For reshape, resize, and transpose, the single tuple argument may be
+replaced with ``n`` integers which will be interpreted as an n-tuple.
+
+.. autosummary::
+   :toctree: generated/
+
+   MaskedArray.flatten
+   MaskedArray.ravel
+   MaskedArray.reshape
+   MaskedArray.resize
+   MaskedArray.squeeze
+   MaskedArray.swapaxes
+   MaskedArray.transpose
+
+
+Item selection and manipulation
+-------------------------------
+
+For array methods that take an *axis* keyword, it defaults to
+:const:`None`. If axis is *None*, then the array is treated as a 1-D
+array. Any other value for *axis* represents the dimension along which
+the operation should proceed.
+
+.. autosummary::
+   :toctree: generated/
+
+   MaskedArray.argsort
+   MaskedArray.choose
+   MaskedArray.compress
+   MaskedArray.diagonal
+   MaskedArray.nonzero
+   MaskedArray.put
+   MaskedArray.repeat
+   MaskedArray.searchsorted
+   MaskedArray.sort
+   MaskedArray.take
+
+
+Calculations
+------------
+
+.. autosummary::
+   :toctree: generated/
+
+   MaskedArray.all
+   MaskedArray.anom
+   MaskedArray.any
+   MaskedArray.argmax
+   MaskedArray.argmin
+   MaskedArray.clip
+   MaskedArray.conj
+   MaskedArray.cumprod
+   MaskedArray.cumsum
+   MaskedArray.mean
+   MaskedArray.min
+   MaskedArray.prod
+   MaskedArray.ptp
+   MaskedArray.round
+   MaskedArray.std
+   MaskedArray.sum
+   MaskedArray.trace
+   MaskedArray.var
+
+
+Arithmetic and comparison operations
+------------------------------------
+
+.. index:: comparison, arithmetic, operation, operator
+
+Comparison operators:
+~~~~~~~~~~~~~~~~~~~~~
+
+.. autosummary::
+   :toctree: generated/
+
+   MaskedArray.__lt__
+   MaskedArray.__le__
+   MaskedArray.__gt__
+   MaskedArray.__ge__
+   MaskedArray.__eq__
+   MaskedArray.__ne__
+
+Truth value of an array (:func:`bool()`):
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. autosummary::
+   :toctree: generated/
+
+   MaskedArray.__nonzero__
+
+
+Arithmetic:
+~~~~~~~~~~~
+
+.. autosummary::
+   :toctree: generated/
+   
+   MaskedArray.__add__
+   MaskedArray.__sub__
+   MaskedArray.__mul__
+   MaskedArray.__div__
+   MaskedArray.__truediv__
+   MaskedArray.__floordiv__
+   MaskedArray.__mod__
+   MaskedArray.__divmod__
+   MaskedArray.__pow__
+   MaskedArray.__lshift__
+   MaskedArray.__rshift__
+   MaskedArray.__and__
+   MaskedArray.__or__
+   MaskedArray.__xor__
+
+
+Arithmetic, in-place:
+~~~~~~~~~~~~~~~~~~~~~
+
+.. autosummary::
+   :toctree: generated/
+   
+   MaskedArray.__iadd__
+   MaskedArray.__isub__
+   MaskedArray.__imul__
+   MaskedArray.__idiv__
+   MaskedArray.__itruediv__
+   MaskedArray.__ifloordiv__
+   MaskedArray.__imod__
+   MaskedArray.__ipow__
+   MaskedArray.__ilshift__
+   MaskedArray.__irshift__
+   MaskedArray.__iand__
+   MaskedArray.__ior__
+   MaskedArray.__ixor__
+
+
+
+Special methods
+---------------
+
+For standard library functions:
+
+.. autosummary::
+   :toctree: generated/
+
+   MaskedArray.__copy__
+   MaskedArray.__deepcopy__
+   MaskedArray.__reduce__
+   MaskedArray.__setstate__
+
+Basic customization:
+
+.. autosummary::
+   :toctree: generated/
+
+   MaskedArray.__new__
+   MaskedArray.__array__
+   MaskedArray.__array_wrap__
+
+Container customization: (see :ref:`Indexing <arrays.indexing>`)
+
+.. autosummary::
+   :toctree: generated/
+
+   MaskedArray.__len__
+   MaskedArray.__getitem__
+   MaskedArray.__setitem__
+   MaskedArray.__getslice__
+   MaskedArray.__setslice__
+   MaskedArray.__contains__
+
+
+
+Specific methods
+----------------
+
+Handling the mask
+~~~~~~~~~~~~~~~~~
+
+The following methods can be used to access information about the mask or to
+manipulate the mask.
+
+.. autosummary::
+   :toctree: generated/
+
+   MaskedArray.harden_mask
+   MaskedArray.soften_mask
+   MaskedArray.unshare_mask
+   MaskedArray.shrink_mask
+
+
+Handling the `fill_value`
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. autosummary::
+   :toctree: generated/
+
+   MaskedArray.get_fill_value
+   MaskedArray.set_fill_value
+
+
+.. autosummary::
+   :toctree: generated/
+
+   MaskedArray.compressed
+   MaskedArray.count
+

Added: trunk/doc/source/reference/maskedarray.generic.rst
===================================================================
--- trunk/doc/source/reference/maskedarray.generic.rst	2009-01-12 21:25:53 UTC (rev 6319)
+++ trunk/doc/source/reference/maskedarray.generic.rst	2009-01-13 21:01:58 UTC (rev 6320)
@@ -0,0 +1,427 @@
+.. currentmodule:: numpy.ma
+
+.. _maskedarray.generic:
+
+
+
+The :mod:`numpy.ma` module
+==========================
+
+Rationale
+---------
+
+Masked arrays are arrays that may have missing or invalid entries.
+The :mod:`numpy.ma` module provides a nearly work-alike replacement for numpy
+that supports data arrays with masks.
+
+
+
+What is a masked array?
+-----------------------
+
+In many circumstances, datasets can be incomplete or tainted by the presence of invalid data. For example, a sensor may have failed to record a data, or
+recorded an invalid value.
+The :mod:`numpy.ma` module provides a convenient way to address this issue, by introducing masked arrays.
+
+A masked array is the combination of a standard :class:`numpy.ndarray` and a mask. A mask is either :attr:`nomask`, indicating that no value of the associated array is invalid, or an array of booleans that determines for each element of the associated array whether the value is valid or not. When an element of the mask is ``False``, the corresponding element of the associated array is valid and is said to be unmasked. When an element of the mask is ``True``, the corresponding element of the associated array is said to be masked (invalid).
+
+The package ensures that masked entries are not used in computations.
+
+As an illustration, let's consider the following dataset::
+
+   >>> import numpy as np
+   >>> x = np.array([1, 2, 3, -1, 5])
+
+We wish to mark the fourth entry as invalid. The easiest is to create a masked
+array::
+
+   >>> mx = ma.masked_array(x, mask=[0, 0, 0, 1, 0])
+
+We can now compute the mean of the dataset, without taking the invalid data into account::
+
+   >>> mx.mean()
+   2.75
+
+
+The :mod:`numpy.ma` module
+--------------------------
+
+
+The main feature of the :mod:`numpy.ma` module is the :class:`~numpy.ma.MaskedArray` class, which is a subclass of :class:`numpy.ndarray`. 
+The class, its attributes and methods are described in more details in the
+:ref:`MaskedArray class <maskedarray.baseclass>` section.
+
+The :mod:`numpy.ma` module can be used as an addition to :mod:`numpy`: ::
+
+   >>> import numpy as np
+   >>> import numpy.ma as ma
+
+To create an array with the second element invalid, we would do::
+
+   >>> y = ma.array([1, 2, 3], mask = [0, 1, 0])
+
+To create a masked array where all values close to 1.e20 are invalid, we would
+do::
+
+   >>> z = masked_values([1.0, 1.e20, 3.0, 4.0], 1.e20)
+
+For a complete discussion of creation methods for masked arrays please see
+section :ref:`Constructing masked arrays <maskedarray.generic.constructing>`.
+
+
+
+
+Using numpy.ma
+==============
+
+.. _maskedarray.generic.constructing:
+
+Constructing masked arrays
+--------------------------
+
+There are several ways to construct a masked array.
+
+* A first possibility is to directly invoke the :class:`MaskedArray` class.
+
+* A second possibility is to use the two masked array constructors,
+  :func:`array` and :func:`masked_array`.
+
+  .. autosummary::
+     :toctree: generated/
+
+     array
+     masked_array
+
+
+* A third option is to take the view of an existing array. In that case, the
+  mask of the view is set to :attr:`nomask` if the array has no named fields,
+  or an array of boolean with the same structure as the array otherwise.::
+
+     >>> x = np.array([1, 2, 3])
+     >>> x.view(ma.MaskedArray)
+     masked_array(data = [1 2 3],
+                  mask = False,
+            fill_value = 999999)
+
+* Yet another possibility is to use any of the following functions:
+
+  .. autosummary::
+     :toctree: generated/
+
+     asarray
+     asanyarray
+     fix_invalid
+     masked_equal
+     masked_greater
+     masked_greater_equal
+     masked_inside
+     masked_invalid
+     masked_less
+     masked_less_equal
+     masked_not_equal
+     masked_object
+     masked_outside
+     masked_values
+     masked_where
+
+
+
+Accessing the data
+------------------
+
+The underlying data of a masked array can be accessed through several ways:
+
+* through the :attr:`data` attribute. The output is a view of the array as 
+  a :class:`numpy.ndarray` or one of its subclasses, depending on the type
+  of the underlying data at the masked array creation.
+
+* through the :meth:`~MaskedArray.__array__` method. The output is then a :class:`numpy.ndarray`.
+
+* by directly taking a view of the masked array as a :class:`numpy.ndarray` or one of its subclass (which is actually what using the :attr:`data` attribute does).
+
+* by using the :func:`getdata` function. 
+
+
+None of these methods is completely satisfactory if some entries have been marked as invalid. As a general rule, invalid data should not be relied on.
+If a representation of the array is needed without any masked entries, it is recommended to fill the array with the :meth:`filled` method.
+
+
+
+Accessing the mask
+------------------
+
+The mask of a masked array is accessible through its :attr:`mask` attribute.
+We must keep in mind that a ``True`` entry in the mask indicates an *invalid* data.
+
+Another possibility is to use the :func:`getmask` and :func:`getmaskarray` functions. :func:`getmask(x)` outputs the mask of ``x`` if ``x`` is a masked array, and the special value :data:`nomask` otherwise.
+:func:`getmaskarray(x)` outputs the mask of ``x`` if ``x`` is a masked array.
+If ``x`` has no invalid entry or is not a masked array, the function outputs  a boolean array of ``False`` with as many elements as ``x``. 
+
+
+
+
+Accessing only the valid entries
+---------------------------------
+
+To retrieve only the valid entries, we can use the inverse of the mask as an index. The inverse of the mask can be calculated with the :func:`numpy.logical_not` function or simply with the ``~`` operator::
+
+   >>> x = ma.array([[1, 2], [3, 4]], mask=[[0, 1], [1, 0]])
+   >>> x[~x.mask]
+   masked_array(data = [1 4],
+                mask = [False False],
+          fill_value = 999999)
+
+Another way to retrieve the valid data is to use the :meth:`compressed` method, which returns a one-dimensional :class:`~numpy.ndarray` (or one of its subclasses, depending on the value of the :attr:`baseclass` attribute)::
+
+   >>> x.compressed
+   array([1, 4])
+
+
+
+Modifying the mask
+------------------
+
+Masking an entry
+~~~~~~~~~~~~~~~~
+
+The recommended way to mark one or several specific entries of a masked array as invalid is to assign the special value :attr:`masked` to them::
+
+   >>> x = ma.array([1, 2, 3])
+   >>> x[0] = ma.masked
+   >>> x
+   masked_array(data = [-- 2 3],
+                mask = [ True False False],
+          fill_value = 999999)
+   >>> y = ma.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
+   >>> y[(0, 1, 2), (1, 2, 0)] = ma.masked
+   >>> y
+   masked_array(data =
+    [[1 -- 3]
+     [4 5 --]
+     [-- 8 9]],
+                mask =
+    [[False  True False]
+     [False False  True]
+     [ True False False]],
+          fill_value = 999999)
+   >>> z = ma.array([1, 2, 3, 4])
+   >>> z[:-2] = ma.masked
+   >>> z
+   masked_array(data = [-- -- 3 4],
+                mask = [ True  True False False],
+          fill_value = 999999)
+
+
+A second possibility is to modify the mask directly, but this usage is    discouraged. 
+
+.. note::
+   When creating a new masked array with a simple, non-structured datatype, the mask is initially set to the special value :attr:`nomask`, that corresponds roughly to the boolean ``False``. Trying to set an element of :attr:`nomask` will fail with a :exc:`TypeError` exception, as a boolean does not support item assignment.
+
+
+All the entries of an array can be masked at once by assigning ``True`` to the mask::
+
+   >>> x = ma.array([1, 2, 3], mask=[0, 0, 1])
+   >>> x.mask = True
+   >>> x
+   masked_array(data = [-- -- --],
+                mask = [ True  True  True],
+          fill_value = 999999)
+
+Finally, specific entries can be masked and/or unmasked by assigning to the mask a sequence of booleans::
+
+   >>> x = ma.array([1, 2, 3])
+   >>> x.mask = [0, 1, 0]
+   >>> x
+   masked_array(data = [1 -- 3],
+                mask = [False  True False],
+          fill_value = 999999)
+
+Unmasking an entry
+~~~~~~~~~~~~~~~~~~
+
+To unmask one or several specific entries, we can just assign one or several new valid values to them::
+
+   >>> x = ma.array([1, 2, 3], mask=[0, 0, 1])
+   >>> x
+   masked_array(data = [1 2 --],
+                mask = [False False  True],
+          fill_value = 999999)
+   >>> x[-1] = 5
+   >>> x
+   masked_array(data = [1 2 5],
+                mask = [False False False],
+          fill_value = 999999)
+
+.. note::
+   Unmasking an entry by direct assignment will not work if the masked array
+   has a *hard* mask, as shown by the :attr:`hardmask`. 
+   This feature was introduced to prevent the overwriting of the mask.
+   To force the unmasking of an entry in such circumstance, the mask has first
+   to be softened with the :meth:`soften_mask` method before the allocation, 
+   and then re-hardened with :meth:`harden_mask`::
+
+      >>> x = ma.array([1, 2, 3], mask=[0, 0, 1])
+      >>> x
+      masked_array(data = [1 2 --],
+                   mask = [False False  True],
+             fill_value = 999999)
+      >>> x[-1] = 5
+      >>> x
+      masked_array(data = [1 2 --],
+                   mask = [False False  True],
+             fill_value = 999999)
+      >>> x.soften_mask()
+      >>> x[-1] = 5
+      >>> x
+      masked_array(data = [1 2 --],
+                   mask = [False False  True],
+             fill_value = 999999)
+      >>> x.soften_mask()
+
+
+To unmask all masked entries of a masked array, the simplest solution is to assign the constant :attr:`nomask` to the mask::
+
+   >>> x = ma.array([1, 2, 3], mask=[0, 0, 1])
+   >>> x
+   masked_array(data = [1 2 --],
+                mask = [False False  True],
+          fill_value = 999999)
+   >>> x.mask = nomask
+   >>> x
+   masked_array(data = [1 2 3],
+                mask = [False False False],
+          fill_value = 999999)
+
+
+
+Indexing and slicing
+--------------------
+
+As a :class:`MaskedArray` is a subclass of :class:`numpy.ndarray`, it inherits its mechanisms for indexing and slicing.
+
+When accessing a single entry of a masked array with no named fields, the output is either a scalar (if the corresponding entry of the mask is ``False``) or the special value :attr:`masked` (if the corresponding entry of the mask is ``True``)::
+
+   >>> x = ma.array([1, 2, 3], mask=[0, 0, 1])
+   >>> x[0]
+   1
+   >>> x[-1]
+   masked_array(data = --,
+                mask = True,
+          fill_value = 1e+20)
+   >>> x[-1] is ma.masked
+   True
+
+If the masked array has named fields, accessing a single entry returns a 
+:class:`numpy.void` object if none of the fields are masked, or a 0d masked array with the same dtype as the initial array if at least one of the fields is masked.
+
+   >>> y = ma.masked_array([(1,2), (3, 4)], 
+   ...                mask=[(0, 0), (0, 1)],
+   ...               dtype=[('a', int), ('b', int)])
+   >>> y[0]
+   (1, 2)
+   >>> y[-1]
+   masked_array(data = (3, --),
+                mask = (False, True),
+          fill_value = (999999, 999999),
+               dtype = [('a', '<i4'), ('b', '<i4')])
+
+
+When accessing a slice, the output is a masked array whose :attr:`data` attribute is a view of the original data, and whose mask is either :attr:`nomask` (if there was no invalid entries in the original array) or a copy of the corresponding slice of the original mask. The copy is required to avoid propagation of any modification of the mask to the original.
+
+   >>> x = ma.array([1, 2, 3, 4, 5], mask=[0, 1, 0, 0, 1])
+   >>> mx = x[:3]
+   >>> mx
+   masked_array(data = [1 -- 3],
+                mask = [False  True False],
+          fill_value = 999999)
+   >>> mx[1] = -1
+   >>> mx
+   masked_array(data = [1 -1 3],
+                mask = [False  True False],
+          fill_value = 999999)
+   >>> x.mask
+   array([False,  True, False, False,  True], dtype=bool)
+   >>> x.data
+   array([ 1, -1,  3,  4,  5])
+
+
+Accessing a field of a masked array with structured datatype returns a :class:`MaskedArray`.
+
+
+
+Operations on masked arrays
+---------------------------
+
+Arithmetic and comparison operations are supported by masked arrays.
+As much as possible, invalid entries of a masked array are not processed, meaning that the corresponding :attr:`data` entries *should* be the same before and after the operation.
+We need to stress that this behavior may not be systematic, that invalid data may actually be affected by the operation in some cases and once again that invalid data should not be relied on.
+
+The :mod:`numpy.ma` module comes with a specific implementation of most
+ufuncs. Unary and binary functions that have a validity domain (such as :func:`~numpy.log` or :func:`~numpy.divide`) return the :data:`masked` constant whenever the input is masked or falls outside the validity domain::
+
+   >>> ma.log([-1, 0, 1, 2])
+   masked_array(data = [-- -- 0.0 0.69314718056],
+                mask = [ True  True False False],
+          fill_value = 1e+20)
+
+Masked arrays also support standard numpy ufuncs. The output is then a masked array. The result of a unary ufunc is masked wherever the input is masked. The result of a binary ufunc is masked wherever any of the input is masked. If the ufunc also returns the optional context output (a 3-element tuple containing the name of the ufunc, its arguments and its domain), the context is processed and entries of the output masked array are masked wherever the corresponding input fall outside the validity domain::
+
+   >>> x = ma.array([-1, 1, 0, 2, 3], mask=[0, 0, 0, 0, 1])
+   >>> np.log(x)
+   masked_array(data = [-- -- 0.0 0.69314718056 --],
+                mask = [ True  True False False  True],
+          fill_value = 1e+20)
+
+
+
+Examples
+========
+
+Data with a given value representing missing data
+-------------------------------------------------
+
+Let's consider a list of elements, ``x``, where values of -9999. represent missing data.
+We wish to compute the average value of the data and the vector of anomalies (deviations from the average)::
+
+   >>> import numpy.ma as ma
+   >>> x = [0.,1.,-9999.,3.,4.]
+   >>> mx = ma.masked_values (x, -9999.)
+   >>> print mx.mean()
+   2.0
+   >>> print mx - mx.mean()
+   [-2.0 -1.0 -- 1.0 2.0]
+   >>> print mx.anom()
+   [-2.0 -1.0 -- 1.0 2.0]
+
+
+Filling in the missing data
+---------------------------
+
+Suppose now that we wish to print that same data, but with the missing values
+replaced by the average value.
+
+   >>> print mx.filled(mx.mean())
+   [ 0.  1.  2.  3.  4.]
+
+
+Numerical operations
+--------------------
+
+Numerical operations can be easily performed without worrying about missing values, dividing by zero, square roots of negative numbers, etc.::
+
+   >>> import numpy as np, numpy.ma as ma
+   >>> x = ma.array([1., -1., 3., 4., 5., 6.], mask=[0,0,0,0,1,0])
+   >>> y = ma.array([1., 2., 0., 4., 5., 6.], mask=[0,0,0,0,0,1])
+   >>> print np.sqrt(x/y)
+   [1.0 -- -- 1.0 -- --]
+ 
+Four values of the output are invalid: the first one comes from taking the square root of a negative number, the second from the division by zero, and the last two where the inputs were masked.
+
+
+Ignoring extreme values
+-----------------------
+
+Let's consider an array ``d`` of random floats between 0 and 1. 
+We wish to compute the average of the values of ``d`` while ignoring any data outside the range [0.1, 0.9]::
+
+   >>> print ma.masked_outside(d, 0.1, 0.9).mean()

Added: trunk/doc/source/reference/maskedarray.rst
===================================================================
--- trunk/doc/source/reference/maskedarray.rst	2009-01-12 21:25:53 UTC (rev 6319)
+++ trunk/doc/source/reference/maskedarray.rst	2009-01-13 21:01:58 UTC (rev 6320)
@@ -0,0 +1,14 @@
+.. _maskedarray:
+
+
+Masked arrays are arrays that may have missing or invalid entries.
+The :mod:`numpy.ma` module provides a nearly work-alike replacement for numpy
+that supports data arrays with masks.
+
+
+.. toctree::
+   :maxdepth: 2
+
+   maskedarray.generic
+   maskedarray.baseclass
+   routines.ma
\ No newline at end of file

Modified: trunk/numpy/ma/core.py
===================================================================
--- trunk/numpy/ma/core.py	2009-01-12 21:25:53 UTC (rev 6319)
+++ trunk/numpy/ma/core.py	2009-01-13 21:01:58 UTC (rev 6320)
@@ -1,19 +1,22 @@
 # pylint: disable-msg=E1002
 """
-MA: a facility for dealing with missing observations
-MA is generally used as a numpy.array look-alike.
-by Paul F. Dubois.
+numpy.ma : a package to handle missing or invalid values.
 
+This package was initially written for numarray by Paul F. Dubois
+at Lawrence Livermore National Laboratory. 
+In 2006, the package was completely rewritten by Pierre Gerard-Marchant
+(University of Georgia) to make the MaskedArray class a subclass of ndarray,
+and to improve support of structured arrays.
+
+
 Copyright 1999, 2000, 2001 Regents of the University of California.
 Released for unlimited redistribution.
-Adapted for numpy_core 2005 by Travis Oliphant and
-(mainly) Paul Dubois.
-
+* Adapted for numpy_core 2005 by Travis Oliphant and (mainly) Paul Dubois.
 * Subclassing of the base ndarray 2006 by Pierre Gerard-Marchant 
   (pgmdevlist_AT_gmail_DOT_com)
 * Improvements suggested by Reggie Dugard (reggie_AT_merfinllc_DOT_com)
 
-:author: Pierre Gerard-Marchant
+.. moduleauthor:: Pierre Gerard-Marchant
 
 
 """
@@ -1106,7 +1109,7 @@
 
 def masked_where(condition, a, copy=True):
     """
-    Return ``a`` as an array masked where ``condition`` is True.
+    Return ``a`` as an array masked where ``condition`` is ``True``.
     Masked values of ``a`` or ``condition`` are kept.
 
     Parameters
@@ -1139,7 +1142,7 @@
 
 def masked_greater(x, value, copy=True):
     """
-    Return the array `x` masked where (x > value).
+    Return the array `x` masked where ``(x > value)``.
     Any value of mask already masked is kept masked.
 
     """
@@ -1147,30 +1150,34 @@
 
 
 def masked_greater_equal(x, value, copy=True):
-    "Shortcut to masked_where, with condition = (x >= value)."
+    "Shortcut to masked_where, with condition ``(x >= value)``."
     return masked_where(greater_equal(x, value), x, copy=copy)
 
 
 def masked_less(x, value, copy=True):
-    "Shortcut to masked_where, with condition = (x < value)."
+    "Shortcut to masked_where, with condition ``(x < value)``."
     return masked_where(less(x, value), x, copy=copy)
 
 
 def masked_less_equal(x, value, copy=True):
-    "Shortcut to masked_where, with condition = (x <= value)."
+    "Shortcut to masked_where, with condition ``(x <= value)``."
     return masked_where(less_equal(x, value), x, copy=copy)
 
 
 def masked_not_equal(x, value, copy=True):
-    "Shortcut to masked_where, with condition = (x != value)."
+    "Shortcut to masked_where, with condition ``(x != value)``."
     return masked_where(not_equal(x, value), x, copy=copy)
 
 
 def masked_equal(x, value, copy=True):
     """
-    Shortcut to masked_where, with condition = (x == value).  For
-    floating point, consider ``masked_values(x, value)`` instead.
+    Shortcut to masked_where, with condition ``(x == value)``.
 
+    See Also
+    --------
+    masked_where : base function
+    masked_values : equivalent function for floats.
+
     """
     # An alternative implementation relies on filling first: probably not needed.
     # d = filled(x, 0)
@@ -1755,7 +1762,8 @@
         return
     #..................................
     def __array_wrap__(self, obj, context=None):
-        """Special hook for ufuncs.
+        """
+        Special hook for ufuncs.
         Wraps the numpy array and sets the mask according to context.
         """
         result = obj.view(type(self))
@@ -1988,7 +1996,8 @@
             ndarray.__setitem__(_data, indx, dindx)
             _mask[indx] = mindx
         return
-    #............................................
+
+
     def __getslice__(self, i, j):
         """x.__getslice__(i, j) <==> x[i:j]
 
@@ -1997,7 +2006,8 @@
 
         """
         return self.__getitem__(slice(i, j))
-    #........................
+
+
     def __setslice__(self, i, j, value):
         """x.__setslice__(i, j, value) <==> x[i:j]=value
 
@@ -2006,7 +2016,8 @@
 
         """
         self.__setitem__(slice(i, j), value)
-    #............................................
+
+
     def __setmask__(self, mask, copy=False):
         """Set the mask.
 
@@ -2107,6 +2118,10 @@
         """
         self._hardmask = False
 
+    hardmask = property(fget=lambda self: self._hardmask,
+                        doc="Hardness of the mask")
+
+
     def unshare_mask(self):
         """Copy the mask and set the sharedmask flag to False.
 
@@ -2115,6 +2130,9 @@
             self._mask = self._mask.copy()
             self._sharedmask = False
 
+    sharedmask = property(fget=lambda self: self._sharedmask,
+                          doc="Share status of the mask (read-only).")
+
     def shrink_mask(self):
         """Reduce a mask to nomask when possible.
 
@@ -2124,6 +2142,10 @@
             self._mask = nomask
 
     #............................................
+
+    baseclass = property(fget= lambda self:self._baseclass,
+                         doc="Class of the underlying data (read-only).")
+    
     def _get_data(self):
         """Return the current data, as a view of the original
         underlying data.
@@ -2179,23 +2201,23 @@
 
 
     def filled(self, fill_value=None):
-        """Return a copy of self._data, where masked values are filled
-        with fill_value.
+        """
+    Return a copy of self, where masked values are filled with `fill_value`.
 
-        If fill_value is None, self.fill_value is used instead.
+    If `fill_value` is None, `self.fill_value` is used instead.
 
-        Notes
-        -----
-        + Subclassing is preserved
-        + The result is NOT a MaskedArray !
+    Notes
+    -----
+    + Subclassing is preserved
+    + The result is NOT a MaskedArray !
 
-        Examples
-        --------
-        >>> x = np.ma.array([1,2,3,4,5], mask=[0,0,1,0,1], fill_value=-999)
-        >>> x.filled()
-        array([1,2,-999,4,-999])
-        >>> type(x.filled())
-        <type 'numpy.ndarray'>
+    Examples
+    --------
+    >>> x = np.ma.array([1,2,3,4,5], mask=[0,0,1,0,1], fill_value=-999)
+    >>> x.filled()
+    array([1,2,-999,4,-999])
+    >>> type(x.filled())
+    <type 'numpy.ndarray'>
 
         """
         m = self._mask
@@ -2502,7 +2524,7 @@
         return self
     #...
     def __ipow__(self, other):
-        "Raise self to the power other, in place"
+        "Raise self to the power other, in place."
         other_data = getdata(other)
         other_mask = getmask(other)
         ndarray.__ipow__(self._data, np.where(self._mask, 1, other_data))
@@ -3728,6 +3750,7 @@
     def toflex(self):
         """
         Transforms a MaskedArray into a flexible-type array with two fields:
+
         * the ``_data`` field stores the ``_data`` part of the array;
         * the ``_mask`` field stores the ``_mask`` part of the array;
 
@@ -4563,7 +4586,8 @@
 outerproduct = outer
 
 def allequal (a, b, fill_value=True):
-    """Return True if all entries of a and b are equal, using
+    """
+    Return True if all entries of a and b are equal, using
     fill_value as a truth value where either or both are masked.
 
     """
@@ -4654,9 +4678,9 @@
     return np.all(d)
 
 #..............................................................................
-def asarray(a, dtype=None):
+def asarray(a, dtype=None, order=None):
     """
-    Convert the input to a masked array.
+    Convert the input `a` to a masked array of the given datatype.
 
     Parameters
     ----------
@@ -4674,24 +4698,35 @@
     -------
     out : ndarray
         MaskedArray interpretation of `a`.  No copy is performed if the input
-        is already an ndarray.  If `a` is a subclass of ndarray, a base
-        class ndarray is returned.
-    Return a as a MaskedArray object of the given dtype.
-    If dtype is not given or None, is is set to the dtype of a.
-    No copy is performed if a is already an array.
-    Subclasses are converted to the base class MaskedArray.
+        is already an ndarray.  If `a` is a subclass of MaskedArray, a base
+        class MaskedArray is returned.
 
     """
     return masked_array(a, dtype=dtype, copy=False, keep_mask=True, subok=False)
 
 def asanyarray(a, dtype=None):
-    """asanyarray(data, dtype) = array(data, dtype, copy=0, subok=1)
+    """
+    Convert the input `a` to a masked array of the given datatype.
+    If `a` is a subclass of MaskedArray, its class is conserved.
 
-    Return a as an masked array.
-    If dtype is not given or None, is is set to the dtype of a.
-    No copy is performed if a is already an array.
-    Subclasses are conserved.
+    Parameters
+    ----------
+    a : array_like
+        Input data, in any form that can be converted to an array.  This
+        includes lists, lists of tuples, tuples, tuples of tuples, tuples
+        of lists and ndarrays.
+    dtype : data-type, optional
+        By default, the data-type is inferred from the input data.
+    order : {'C', 'F'}, optional
+        Whether to use row-major ('C') or column-major ('FORTRAN') memory
+        representation.  Defaults to 'C'.
 
+    Returns
+    -------
+    out : ndarray
+        MaskedArray interpretation of `a`.  No copy is performed if the input
+        is already an ndarray.
+
     """
     return masked_array(a, dtype=dtype, copy=False, keep_mask=True, subok=True)
 

Modified: trunk/numpy/ma/extras.py
===================================================================
--- trunk/numpy/ma/extras.py	2009-01-12 21:25:53 UTC (rev 6319)
+++ trunk/numpy/ma/extras.py	2009-01-13 21:01:58 UTC (rev 6320)
@@ -45,22 +45,19 @@
 #...............................................................................
 def issequence(seq):
     """Is seq a sequence (ndarray, list or tuple)?"""
-    if isinstance(seq, ndarray):
+    if isinstance(seq, (ndarray, tuple, list)):
         return True
-    elif isinstance(seq, tuple):
-        return True
-    elif isinstance(seq, list):
-        return True
     return False
 
 def count_masked(arr, axis=None):
-    """Count the number of masked elements along the given axis.
+    """
+    Count the number of masked elements along the given axis.
 
     Parameters
     ----------
-        axis : int, optional
-            Axis along which to count.
-            If None (default), a flattened version of the array is used.
+    axis : int, optional
+        Axis along which to count.
+        If None (default), a flattened version of the array is used.
 
     """
     m = getmaskarray(arr)
@@ -252,7 +249,8 @@
 
 
 def average(a, axis=None, weights=None, returned=False):
-    """Average the array over the given axis.
+    """
+    Average the array over the given axis.
 
     Parameters
     ----------
@@ -440,10 +438,10 @@
 #..............................................................................
 def compress_rowcols(x, axis=None):
     """
-    Suppress the rows and/or columns of a 2D array that contains
+    Suppress the rows and/or columns of a 2D array that contain
     masked values.
 
-    The suppression behavior is selected with the `axis`parameter.
+    The suppression behavior is selected with the `axis` parameter.
 
         - If axis is None, rows and columns are suppressed.
         - If axis is 0, only rows are suppressed.
@@ -482,13 +480,15 @@
     return x._data[idxr][:,idxc]
 
 def compress_rows(a):
-    """Suppress whole rows of a 2D array that contain masked values.
+    """
+    Suppress whole rows of a 2D array that contain masked values.
 
     """
     return compress_rowcols(a, 0)
 
 def compress_cols(a):
-    """Suppress whole columnss of a 2D array that contain masked values.
+    """
+    Suppress whole columns of a 2D array that contain masked values.
 
     """
     return compress_rowcols(a, 1)
@@ -530,30 +530,35 @@
     return a
 
 def mask_rows(a, axis=None):
-    """Mask whole rows of a 2D array that contain masked values.
+    """
+    Mask whole rows of a 2D array that contain masked values.
 
     Parameters
     ----------
-        axis : int, optional
-            Axis along which to perform the operation.
-            If None, applies to a flattened version of the array.
+    axis : int, optional
+        Axis along which to perform the operation.
+        If None, applies to a flattened version of the array.
+
     """
     return mask_rowcols(a, 0)
 
 def mask_cols(a, axis=None):
-    """Mask whole columns of a 2D array that contain masked values.
+    """
+    Mask whole columns of a 2D array that contain masked values.
 
     Parameters
     ----------
-        axis : int, optional
-            Axis along which to perform the operation.
-            If None, applies to a flattened version of the array.
+    axis : int, optional
+        Axis along which to perform the operation.
+        If None, applies to a flattened version of the array.
+
     """
     return mask_rowcols(a, 1)
 
 
 def dot(a,b, strict=False):
-    """Return the dot product of two 2D masked arrays a and b.
+    """
+    Return the dot product of two 2D masked arrays a and b.
 
     Like the generic numpy equivalent, the product sum is over the last
     dimension of a and the second-to-last dimension of b.  If strict is True,
@@ -584,24 +589,25 @@
 
 #...............................................................................
 def ediff1d(array, to_end=None, to_begin=None):
-    """Return the differences between consecutive elements of an
+    """
+    Return the differences between consecutive elements of an
     array, possibly with prefixed and/or appended values.
 
     Parameters
     ----------
-        array : {array}
-            Input array,  will be flattened before the difference is taken.
-        to_end : {number}, optional
-            If provided, this number will be tacked onto the end of the returned
-            differences.
-        to_begin : {number}, optional
-            If provided, this number will be taked onto the beginning of the
-            returned differences.
+    array : {array}
+        Input array,  will be flattened before the difference is taken.
+    to_end : {number}, optional
+        If provided, this number will be tacked onto the end of the returned
+        differences.
+    to_begin : {number}, optional
+        If provided, this number will be taked onto the beginning of the
+        returned differences.
 
     Returns
     -------
-          ed : {array}
-            The differences. Loosely, this will be (ary[1:] - ary[:-1]).
+    ed : {array}
+        The differences. Loosely, this will be (ary[1:] - ary[:-1]).
 
     """
     a = masked_array(array, copy=True)
@@ -747,7 +753,8 @@
 
 
 def corrcoef(x, y=None, rowvar=True, bias=False, allow_masked=True):
-    """The correlation coefficients formed from the array x, where the
+    """
+    The correlation coefficients formed from the array x, where the
     rows are the observations, and the columns are variables.
 
     corrcoef(x,y) where x and y are 1d arrays is the same as
@@ -818,7 +825,8 @@
 #####--------------------------------------------------------------------------
 
 class MAxisConcatenator(AxisConcatenator):
-    """Translate slice objects to concatenation along an axis.
+    """
+    Translate slice objects to concatenation along an axis.
 
     """
 
@@ -877,11 +885,13 @@
         return self._retval(res)
 
 class mr_class(MAxisConcatenator):
-    """Translate slice objects to concatenation along the first axis.
+    """
+    Translate slice objects to concatenation along the first axis.
 
-    For example:
-        >>> np.ma.mr_[np.ma.array([1,2,3]), 0, 0, np.ma.array([4,5,6])]
-        array([1, 2, 3, 0, 0, 4, 5, 6])
+    Examples
+    --------
+    >>> np.ma.mr_[np.ma.array([1,2,3]), 0, 0, np.ma.array([4,5,6])]
+    array([1, 2, 3, 0, 0, 4, 5, 6])
 
     """
     def __init__(self):
@@ -894,7 +904,8 @@
 #####--------------------------------------------------------------------------
 
 def flatnotmasked_edges(a):
-    """Find the indices of the first and last not masked values in a
+    """
+    Find the indices of the first and last not masked values in a
     1D masked array.  If all values are masked, returns None.
 
     """
@@ -907,8 +918,10 @@
     else:
         return None
 
+
 def notmasked_edges(a, axis=None):
-    """Find the indices of the first and last not masked values along
+    """
+    Find the indices of the first and last not masked values along
     the given axis in a masked array.
 
     If all values are masked, return None.  Otherwise, return a list
@@ -917,9 +930,10 @@
 
     Parameters
     ----------
-        axis : int, optional
-            Axis along which to perform the operation.
-            If None, applies to a flattened version of the array.
+    axis : int, optional
+        Axis along which to perform the operation.
+        If None, applies to a flattened version of the array.
+
     """
     a = asarray(a)
     if axis is None or a.ndim == 1:
@@ -929,8 +943,10 @@
     return [tuple([idx[i].min(axis).compressed() for i in range(a.ndim)]),
             tuple([idx[i].max(axis).compressed() for i in range(a.ndim)]),]
 
+
 def flatnotmasked_contiguous(a):
-    """Find contiguous unmasked data in a flattened masked array.
+    """
+    Find contiguous unmasked data in a flattened masked array.
 
     Return a sorted sequence of slices (start index, end index).
 
@@ -950,22 +966,22 @@
     return result
 
 def notmasked_contiguous(a, axis=None):
-    """Find contiguous unmasked data in a masked array along the given
-    axis.
+    """
+    Find contiguous unmasked data in a masked array along the given axis.
 
     Parameters
     ----------
-        axis : int, optional
-            Axis along which to perform the operation.
-            If None, applies to a flattened version of the array.
+    axis : int, optional
+        Axis along which to perform the operation.
+        If None, applies to a flattened version of the array.
 
     Returns
     -------
-        A sorted sequence of slices (start index, end index).
+    A sorted sequence of slices (start index, end index).
 
     Notes
     -----
-        Only accepts 2D arrays at most.
+    Only accepts 2D arrays at most.
 
     """
     a = asarray(a)



More information about the Numpy-svn mailing list