[Numpy-svn] [numpy/numpy] b272bc: Remove maskna API from ndarray, and all (and only)...

GitHub noreply@github....
Thu Jun 21 03:37:05 CDT 2012


  Branch: refs/heads/master
  Home:   https://github.com/numpy/numpy
  Commit: b272bc605ce7784be5b3edb13ad7afe22b04e71f
      https://github.com/numpy/numpy/commit/b272bc605ce7784be5b3edb13ad7afe22b04e71f
  Author: Nathaniel J. Smith <njs@pobox.com>
  Date:   2012-06-16 (Sat, 16 Jun 2012)

  Changed paths:
    R doc/source/reference/arrays.maskna.rst
    M doc/source/reference/arrays.rst
    M doc/source/reference/c-api.array.rst
    R doc/source/reference/c-api.maskna.rst
    M doc/source/reference/c-api.rst
    M doc/source/reference/routines.polynomials.classes.rst
    M doc/source/reference/routines.rst
    M numpy/add_newdocs.py
    M numpy/core/_methods.py
    M numpy/core/arrayprint.py
    M numpy/core/code_generators/genapi.py
    M numpy/core/code_generators/numpy_api.py
    M numpy/core/fromnumeric.py
    M numpy/core/function_base.py
    M numpy/core/include/numpy/ndarraytypes.h
    M numpy/core/include/numpy/ufuncobject.h
    M numpy/core/numeric.py
    M numpy/core/numerictypes.py
    M numpy/core/setup.py
    M numpy/core/src/multiarray/array_assign.h
    M numpy/core/src/multiarray/array_assign_array.c
    M numpy/core/src/multiarray/array_assign_scalar.c
    M numpy/core/src/multiarray/arrayobject.c
    M numpy/core/src/multiarray/arraytypes.c.src
    R numpy/core/src/multiarray/boolean_ops.c.src
    R numpy/core/src/multiarray/boolean_ops.h
    M numpy/core/src/multiarray/buffer.c
    M numpy/core/src/multiarray/calculation.c
    M numpy/core/src/multiarray/common.c
    M numpy/core/src/multiarray/common.h
    M numpy/core/src/multiarray/conversion_utils.c
    M numpy/core/src/multiarray/convert.c
    M numpy/core/src/multiarray/convert_datatype.c
    M numpy/core/src/multiarray/ctors.c
    M numpy/core/src/multiarray/ctors.h
    M numpy/core/src/multiarray/datetime.c
    M numpy/core/src/multiarray/dtype_transfer.c
    M numpy/core/src/multiarray/einsum.c.src
    M numpy/core/src/multiarray/flagsobject.c
    M numpy/core/src/multiarray/getset.c
    M numpy/core/src/multiarray/item_selection.c
    M numpy/core/src/multiarray/item_selection.h
    M numpy/core/src/multiarray/iterators.c
    M numpy/core/src/multiarray/iterators.h
    M numpy/core/src/multiarray/mapping.c
    M numpy/core/src/multiarray/methods.c
    M numpy/core/src/multiarray/multiarraymodule.c
    M numpy/core/src/multiarray/multiarraymodule_onefile.c
    R numpy/core/src/multiarray/na_mask.c
    R numpy/core/src/multiarray/na_mask.h
    R numpy/core/src/multiarray/na_object.c
    R numpy/core/src/multiarray/na_object.h
    M numpy/core/src/multiarray/nditer_api.c
    M numpy/core/src/multiarray/nditer_constr.c
    M numpy/core/src/multiarray/nditer_impl.h
    M numpy/core/src/multiarray/nditer_pywrap.c
    M numpy/core/src/multiarray/reduction.c
    M numpy/core/src/multiarray/reduction.h
    M numpy/core/src/multiarray/scalarapi.c
    M numpy/core/src/multiarray/sequence.c
    M numpy/core/src/multiarray/shape.c
    M numpy/core/src/private/lowlevel_strided_loops.h
    M numpy/core/src/umath/ufunc_object.c
    M numpy/core/src/umath/ufunc_type_resolution.c
    M numpy/core/tests/test_api.py
    M numpy/core/tests/test_indexerrors.py
    R numpy/core/tests/test_maskna.py
    M numpy/core/tests/test_multiarray.py
    R numpy/core/tests/test_na.py
    M numpy/core/tests/test_nditer.py
    M numpy/core/tests/test_numeric.py
    M numpy/core/tests/test_regression.py
    M numpy/lib/function_base.py
    M numpy/lib/twodim_base.py
    M numpy/polynomial/chebyshev.py
    M numpy/polynomial/hermite.py
    M numpy/polynomial/hermite_e.py
    M numpy/polynomial/laguerre.py
    M numpy/polynomial/legendre.py
    M numpy/polynomial/polynomial.py
    M numpy/polynomial/tests/test_chebyshev.py
    M numpy/polynomial/tests/test_hermite.py
    M numpy/polynomial/tests/test_hermite_e.py
    M numpy/polynomial/tests/test_laguerre.py
    M numpy/polynomial/tests/test_legendre.py
    M numpy/polynomial/tests/test_polynomial.py
    M numpy/testing/utils.py

  Log Message:
  -----------
  Remove maskna API from ndarray, and all (and only) the code supporting it

The original masked-NA-NEP branch contained a large number of changes
in addition to the core NA support. For example:
 - ufunc.__call__ support for where= argument
 - nditer support for arbitrary masks (in support of where=)
 - ufunc.reduce support for simultaneous reduction over multiple axes
 - a new "array assignment API"
 - ndarray.diagonal() returning a view in all cases
 - bug-fixes in __array_priority__ handling
 - datetime test changes
etc. There's no consensus yet on what should be done with the
maskna-related part of this branch, but the rest is generally useful
and uncontroversial, so the goal of this branch is to identify exactly
which code changes are involved in maskna support.

The basic strategy used to create this patch was:
 - Remove the new masking-related fields from ndarray, so no arrays
   are masked
 - Go through and remove all the code that this makes
   dead/inaccessible/irrelevant, in a largely mechanical fashion. So
   for example, if I saw 'if (PyArray_HASMASK(a)) { ... }' then that
   whole block was obviously just dead code if no arrays have masks,
   and I removed it. Likewise for function arguments like skipna that
   are useless if there aren't any NAs to skip.

This changed the signature of a number of functions that were newly
exposed in the numpy public API. I've removed all such functions from
the public API, since releasing them with the NA-less signature in 1.7
would create pointless compatibility hassles later if and when we add
back the NA-related functionality. Most such functions are removed by
this commit; the exception is PyArray_ReduceWrapper, which requires
more extensive surgery, and will be handled in followup commits.

I also removed the new ndarray.setasflat method. Reason: a comment
noted that the only reason this was added was to allow easier testing
of one branch of PyArray_CopyAsFlat. That branch is now the main
branch, so that isn't an issue. Nonetheless this function is arguably
useful, so perhaps it should have remained, but I judged that since
numpy's API is already hairier than we would like, it's not a good
idea to add extra hair "just in case". (Also AFAICT the test for this
method in test_maskna was actually incorrect, as noted here:
   https://github.com/njsmith/numpyNEP/blob/master/numpyNEP.py
so I'm not confident that it ever worked in master, though I haven't
had a chance to follow-up on this.)

I also removed numpy.count_reduce_items, since without skipna it
became trivial.

I believe that these are the only exceptions to the "remove dead code"
strategy.


  Commit: 605c2b45caa8838716b736e322f9e1b5d347daba
      https://github.com/numpy/numpy/commit/605c2b45caa8838716b736e322f9e1b5d347daba
  Author: Nathaniel J. Smith <njs@pobox.com>
  Date:   2012-06-16 (Sat, 16 Jun 2012)

  Changed paths:
    R numpy/core/src/multiarray/reduction.c
    R numpy/core/src/multiarray/reduction.h
    A numpy/core/src/umath/reduction.c
    A numpy/core/src/umath/reduction.h

  Log Message:
  -----------
  Move reduction.{c,h} from multiarray/ to umath/

This is done as a separate commit to make sure git can track the
change. This commit will not build. See next commit for actual changes
and rationale.


  Commit: 3626d0c4fe510d615ef3e5ef3cf4ed2bfb52b53e
      https://github.com/numpy/numpy/commit/3626d0c4fe510d615ef3e5ef3cf4ed2bfb52b53e
  Author: Nathaniel J. Smith <njs@pobox.com>
  Date:   2012-06-16 (Sat, 16 Jun 2012)

  Changed paths:
    M numpy/add_newdocs.py
    M numpy/core/SConscript
    M numpy/core/code_generators/cversions.txt
    M numpy/core/code_generators/genapi.py
    M numpy/core/code_generators/numpy_api.py
    M numpy/core/include/numpy/ndarraytypes.h
    M numpy/core/setup.py
    M numpy/core/src/multiarray/item_selection.c
    M numpy/core/src/multiarray/item_selection.h
    M numpy/core/src/multiarray/multiarraymodule.c
    M numpy/core/src/multiarray/multiarraymodule_onefile.c
    M numpy/core/src/umath/reduction.c
    M numpy/core/src/umath/reduction.h
    M numpy/core/src/umath/ufunc_object.c
    M numpy/core/src/umath/umathmodule_onefile.c
    M numpy/core/tests/test_numeric.py

  Log Message:
  -----------
  Remove PyArray_ReduceWrapper from public API

There are two reasons to want to keep PyArray_ReduceWrapper out of the
public multiarray API:
 - Its signature is likely to change if/when masked arrays are added
 - It is essentially a wrapper for array->scalar transformations
   (*not* just reductions as its name implies -- the whole reason it
   is in multiarray.so in the first place is to support count_nonzero,
   which is not actually a reduction!). It provides some nice
   conveniences (like making it easy to apply such functions to
   multiple axes simultaneously), but, we already have a general
   mechanism for writing array->scalar transformations -- generalized
   ufuncs. We do not want to have two independent, redundant
   implementations of this functionality, one in multiarray and one in
   umath! So in the long run we should add these nice features to the
   generalized ufunc machinery. And in the short run, we shouldn't add
   it to the public API and commit ourselves to supporting it.

However, simply removing it from numpy_api.py is not easy, because
this code was used in both multiarray and umath. This commit:
 - Moves ReduceWrapper and supporting code to umath/, and makes
   appropriate changes (e.g. renaming it to PyUFunc_ReduceWrapper and
   cleaning up the header files).
 - Reverts numpy.count_nonzero to its previous implementation, so that
   it loses the new axis= and keepdims= arguments. This is
   unfortunate, but this change isn't so urgent that it's worth tying
   our APIs in knots forever. (Perhaps in the future it can become a
   generalized ufunc.)


  Commit: 134174c9265dd87ea802c89cac7a89478e3184f4
      https://github.com/numpy/numpy/commit/134174c9265dd87ea802c89cac7a89478e3184f4
  Author: Travis E. Oliphant <teoliphant@gmail.com>
  Date:   2012-06-21 (Thu, 21 Jun 2012)

  Changed paths:
    R doc/source/reference/arrays.maskna.rst
    M doc/source/reference/arrays.rst
    M doc/source/reference/c-api.array.rst
    R doc/source/reference/c-api.maskna.rst
    M doc/source/reference/c-api.rst
    M doc/source/reference/routines.polynomials.classes.rst
    M doc/source/reference/routines.rst
    M numpy/add_newdocs.py
    M numpy/core/SConscript
    M numpy/core/_methods.py
    M numpy/core/arrayprint.py
    M numpy/core/code_generators/cversions.txt
    M numpy/core/code_generators/genapi.py
    M numpy/core/code_generators/numpy_api.py
    M numpy/core/fromnumeric.py
    M numpy/core/function_base.py
    M numpy/core/include/numpy/ndarraytypes.h
    M numpy/core/include/numpy/ufuncobject.h
    M numpy/core/numeric.py
    M numpy/core/numerictypes.py
    M numpy/core/setup.py
    M numpy/core/src/multiarray/array_assign.h
    M numpy/core/src/multiarray/array_assign_array.c
    M numpy/core/src/multiarray/array_assign_scalar.c
    M numpy/core/src/multiarray/arrayobject.c
    M numpy/core/src/multiarray/arraytypes.c.src
    R numpy/core/src/multiarray/boolean_ops.c.src
    R numpy/core/src/multiarray/boolean_ops.h
    M numpy/core/src/multiarray/buffer.c
    M numpy/core/src/multiarray/calculation.c
    M numpy/core/src/multiarray/common.c
    M numpy/core/src/multiarray/common.h
    M numpy/core/src/multiarray/conversion_utils.c
    M numpy/core/src/multiarray/convert.c
    M numpy/core/src/multiarray/convert_datatype.c
    M numpy/core/src/multiarray/ctors.c
    M numpy/core/src/multiarray/ctors.h
    M numpy/core/src/multiarray/datetime.c
    M numpy/core/src/multiarray/dtype_transfer.c
    M numpy/core/src/multiarray/einsum.c.src
    M numpy/core/src/multiarray/flagsobject.c
    M numpy/core/src/multiarray/getset.c
    M numpy/core/src/multiarray/item_selection.c
    M numpy/core/src/multiarray/item_selection.h
    M numpy/core/src/multiarray/iterators.c
    M numpy/core/src/multiarray/iterators.h
    M numpy/core/src/multiarray/mapping.c
    M numpy/core/src/multiarray/methods.c
    M numpy/core/src/multiarray/multiarraymodule.c
    M numpy/core/src/multiarray/multiarraymodule_onefile.c
    R numpy/core/src/multiarray/na_mask.c
    R numpy/core/src/multiarray/na_mask.h
    R numpy/core/src/multiarray/na_object.c
    R numpy/core/src/multiarray/na_object.h
    M numpy/core/src/multiarray/nditer_api.c
    M numpy/core/src/multiarray/nditer_constr.c
    M numpy/core/src/multiarray/nditer_impl.h
    M numpy/core/src/multiarray/nditer_pywrap.c
    R numpy/core/src/multiarray/reduction.c
    R numpy/core/src/multiarray/reduction.h
    M numpy/core/src/multiarray/scalarapi.c
    M numpy/core/src/multiarray/sequence.c
    M numpy/core/src/multiarray/shape.c
    M numpy/core/src/private/lowlevel_strided_loops.h
    A numpy/core/src/umath/reduction.c
    A numpy/core/src/umath/reduction.h
    M numpy/core/src/umath/ufunc_object.c
    M numpy/core/src/umath/ufunc_type_resolution.c
    M numpy/core/src/umath/umathmodule_onefile.c
    M numpy/core/tests/test_api.py
    M numpy/core/tests/test_indexerrors.py
    R numpy/core/tests/test_maskna.py
    M numpy/core/tests/test_multiarray.py
    R numpy/core/tests/test_na.py
    M numpy/core/tests/test_nditer.py
    M numpy/core/tests/test_numeric.py
    M numpy/core/tests/test_regression.py
    M numpy/lib/function_base.py
    M numpy/lib/twodim_base.py
    M numpy/polynomial/chebyshev.py
    M numpy/polynomial/hermite.py
    M numpy/polynomial/hermite_e.py
    M numpy/polynomial/laguerre.py
    M numpy/polynomial/legendre.py
    M numpy/polynomial/polynomial.py
    M numpy/polynomial/tests/test_chebyshev.py
    M numpy/polynomial/tests/test_hermite.py
    M numpy/polynomial/tests/test_hermite_e.py
    M numpy/polynomial/tests/test_laguerre.py
    M numpy/polynomial/tests/test_legendre.py
    M numpy/polynomial/tests/test_polynomial.py
    M numpy/testing/utils.py

  Log Message:
  -----------
  Merge pull request #297 from njsmith/separate-maskna

Split maskna support out of mainline into a branch


Compare: https://github.com/numpy/numpy/compare/651ef74c4ebe...134174c9265d


More information about the Numpy-svn mailing list