[Numpy-discussion] Time for beta1 of NumPy 1.0

Sasha ndarray at mac.com
Fri Jun 30 17:21:21 CDT 2006


"Software developers also use coverage testing in concert with
testsuites, to make sure software is actually good enough for a
release. " -- Gcov Manual

I think if we can improve the test coverage, it will speak volumes
about the quality of numpy.
Does anyone know if it is possible to instrument numpy libraries
without having to instrument python itself?

It would be nice to make the coverage reports easily available either
by including a generating script with the source distribution or by
publishing the reports for the releases.

On 6/30/06, Sasha <ndarray at mac.com> wrote:
> It is not as bad as I thought, but there is certainly room for improvement.
>
> File `numpy/core/src/multiarraymodule.c'
> Lines executed:63.56% of 3290
>
> File `numpy/core/src/arrayobject.c'
> Lines executed:59.70% of 5280
>
> File `numpy/core/src/scalartypes.inc.src'
> Lines executed:31.67% of 963
>
> File `numpy/core/src/arraytypes.inc.src'
> Lines executed:47.35% of 868
>
> File `numpy/core/src/arraymethods.c'
> Lines executed:57.65% of 739
>
>
>
> On 6/30/06, Sasha <ndarray at mac.com> wrote:
> > As soon as I sent out my 10% estimate, I realized that someone will
> > challenge it with a python level coverage statistics.  My main concern
> > is not what fraction of numpy functions is called by unit tests, but
> > what fraction of special cases in the C code is exercised.  I am not
> > sure that David's statistics even answers the first question - I would
> > guess it only counts  statements in the pure python methods and
> > ignores methods implemented in C.
> >
> > Can someone post C-level statistics from gcov
> > <http://gcc.gnu.org/onlinedocs/gcc/Gcov.html> or a similar tool?
> >
> > On 6/30/06, David M. Cooke <cookedm at physics.mcmaster.ca> wrote:
> > > On Fri, 30 Jun 2006 12:35:35 -0400
> > > Sasha <ndarray at mac.com> wrote:
> > >
> > > > On 6/30/06, Fernando Perez <fperez.net at gmail.com> wrote:
> > > > > ...
> > > > > Besides, decent unit tests will catch these problems.  We all know
> > > > > that every scientific code in existence is unit tested to the smallest
> > > > > routine, so this shouldn't be a problem for anyone.
> > > >
> > > > Is this a joke? Did anyone ever measured the coverage of numpy
> > > > unittests? I would be surprized if it was more than 10%.
> > >
> > > A very quick application of the coverage module, available at
> > > http://www.garethrees.org/2001/12/04/python-coverage/
> > > gives me 41%:
> > >
> > > Name                            Stmts   Exec  Cover
> > > ---------------------------------------------------
> > > numpy                              25     20    80%
> > > numpy._import_tools               235    175    74%
> > > numpy.add_newdocs                   2      2   100%
> > > numpy.core                         28     26    92%
> > > numpy.core.__svn_version__          1      1   100%
> > > numpy.core._internal               99     48    48%
> > > numpy.core.arrayprint             251     92    36%
> > > numpy.core.defchararray           221     58    26%
> > > numpy.core.defmatrix              259    186    71%
> > > numpy.core.fromnumeric            319    153    47%
> > > numpy.core.info                     3      3   100%
> > > numpy.core.ma                    1612   1145    71%
> > > numpy.core.memmap                  64     14    21%
> > > numpy.core.numeric                323    138    42%
> > > numpy.core.numerictypes           236    204    86%
> > > numpy.core.records                272     32    11%
> > > numpy.dft                           6      4    66%
> > > numpy.dft.fftpack                 128     31    24%
> > > numpy.dft.helper                   35     32    91%
> > > numpy.dft.info                      3      3   100%
> > > numpy.distutils                    13      9    69%
> > > numpy.distutils.__version__         4      4   100%
> > > numpy.distutils.ccompiler         296     49    16%
> > > numpy.distutils.exec_command      409     27     6%
> > > numpy.distutils.info                2      2   100%
> > > numpy.distutils.log                37     18    48%
> > > numpy.distutils.misc_util         945    174    18%
> > > numpy.distutils.unixccompiler      34     11    32%
> > > numpy.dual                         41     27    65%
> > > numpy.f2py.info                     2      2   100%
> > > numpy.lib                          30     28    93%
> > > numpy.lib.arraysetops             121     59    48%
> > > numpy.lib.function_base           501     70    13%
> > > numpy.lib.getlimits                76     61    80%
> > > numpy.lib.index_tricks            223     56    25%
> > > numpy.lib.info                      4      4   100%
> > > numpy.lib.machar                  174    154    88%
> > > numpy.lib.polynomial              357     52    14%
> > > numpy.lib.scimath                  51     19    37%
> > > numpy.lib.shape_base              220     24    10%
> > > numpy.lib.twodim_base              77     51    66%
> > > numpy.lib.type_check              110     75    68%
> > > numpy.lib.ufunclike                37     24    64%
> > > numpy.lib.utils                    42     23    54%
> > > numpy.linalg                        5      3    60%
> > > numpy.linalg.info                   2      2   100%
> > > numpy.linalg.linalg               440     71    16%
> > > numpy.random                       10      6    60%
> > > numpy.random.info                   4      4   100%
> > > numpy.testing                       3      3   100%
> > > numpy.testing.info                  2      2   100%
> > > numpy.testing.numpytest           430    214    49%
> > > numpy.testing.utils               151     62    41%
> > > numpy.version                       7      7   100%
> > > ---------------------------------------------------
> > > TOTAL                            8982   3764    41%
> > >
> > > (I filtered out all the *.tests.* modules). Note that you have to import
> > > numpy after starting the coverage, because we use a lot of module-level code
> > > that wouldn't be caught otherwise.
> > >
> > > --
> > > |>|\/|<
> > > /--------------------------------------------------------------------------\
> > > |David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
> > > |cookedm at physics.mcmaster.ca
> > >
> >
>




More information about the Numpy-discussion mailing list