[Scipy-tickets] [SciPy] #745: scipy.stats.distribution - tests and bugfixes
SciPy
scipy-tickets@scipy....
Thu Oct 2 08:13:17 CDT 2008
#745: scipy.stats.distribution - tests and bugfixes
-------------------------+--------------------------------------------------
Reporter: josefpktd | Owner: somebody
Type: defect | Status: new
Priority: normal | Milestone:
Component: scipy.stats | Version: devel
Severity: normal | Keywords:
-------------------------+--------------------------------------------------
For original mailing list post see
http://projects.scipy.org/pipermail/scipy-dev/2008-September/009757.html
I spend quite some time hunting and killing bugs in scipy.stats and
learning more about scipy/numpy.
In the attachment are the scipy.stats.distribution.py file with my
corrections and comments, two nosetest files and one messy python script
test file.
I was working off the trunk at revision 4741, then I merged (with bzr and
by hand) 2 change sets, 4752 and 4753, into my
scipy.stats.distributions.py. All current tests and changes are based on
this file. I did not update any other files, since they did not seem
directly relevant. (Also, it is the first time that I used bazar for
merging, but I assume I did it correctly.) NumPy version 1.2.0rc2
I gave up writing individual bug reports, however, I left many comments in
the distributions.py file and in some cases one-line testcases, since this
was the easiest way for me to debug, without having to reload modules all
the time.
In terms of importance and sequence, the bug fix categories are roughly:
* nin correction for vectorize with *args: without this, some methods
raise exceptions, e.g. _ppf (I think after many shaky earlier attempts, I
found the correct solution)
* _ppf did not handle infinite support properly, and requires a .tolist
in call to `place` (no idea why - casting error in `place`, but this
works). Also I added a conditional statement to prevent calls when the
argument list is empty, i.e. [array([]),....].
* _drv2_moment(self, n, *args) was completely broken, it didn't even have
a return. It is the base for generic moment calculations.
* _drv2_ppfsingle: the bisection algorithm had a corner solution wrong
* def moment: for 2nd moment returned the mean (both discrete and
continuous distribution
individual fixes:
* recipinvgauss_gen: rvs added, didn't work before
* randint: -1 was missing in _ppf, definitions differ from those in
description pdf
* dlaplace: _ppf gave wrong results in tests, correction by Per.Brodtkorb
in mailing list, but the generic _ppf works for this case also
* small enhancement proposal (not implemented): include missing cases in
the support of the function in cdf, ppf, ...
The above is all for discrete distribution, for the continuous
distributions, I also did the nin correction now moved, I think correctly,
to the __init__ method.
== Test of Basic Properties ==
The first test just tests for basic properties and their consistency, e.g.
cdf, pmf, ppf, stats and moments and the internal methods e.g. _ppf. The
tests check quite a bit for internal consistency (private methods) since I
used the tests for debugging and trying out fixes. I also added some basic
tests for survival and inverse survival functions which caught an error in
the ppf bisection algorithm. The test functions are now split into public
and private methods.
{{{
>python C:\Programs\Python25\Scripts\nosetests-script.py -s
test_discrete_basic.py
...
}}}
with current trunk (rev4741), I get for this test (approximately since
random):
{{{
(copied from email- did not rerun)
Ran 112 tests in 0.140s
FAILED (errors=41, failures=15)
}}}
with scipy 0.6.0 I get:
{{{
Ran 132 tests in 0.797s
FAILED (errors=65, failures=13)
}}}
after the bugfixes, I get:
{{{
Ran 132 tests in 5.484s
FAILED (failures=3)
}}}
remaining failures:
* dlaplace has an error in the sample variance comparison
* dlaplace and zipf have a error in the generic_moment test which as
Per.Brodtkorb pointed out should be a private methods since in some cases
the limits of the support distribution are (by design) not correctly
initialized yet.
== Chisquare Test ==
The second test is a chisquare test for the random variables to be close
to the theoretical distribution as defined by the .cdf method.
with current trunk I get for this test:
{{{
>python C:\Programs\Python25\Scripts\nosetests-script.py -s
test_ndiscrete.py
or python test_ndiscrete.py
Ran 12 tests in 0.453s
FAILED (errors=5)
}}}
after patching scipy.stats.distribution the only test failure left are
in ``logser`` and ``hypergeom`` (for some parameters), where I think the
numpy.random numbers are wrong
== scipy.stats tests: no failures ==
running the current trunk test for scipy.stats on the pre- and post-patch
distributions.py gives no errors:
{{{
>>> from scipy import stats
>>> stats.test()
Running unit tests for scipy.stats
NumPy version 1.2.0rc2
NumPy is installed in C:\Programs\Python25\lib\site-packages\numpySciPy
version 0.7.0.dev
SciPy is installed in C:\Josef\_progs\virtualpy25\envscipy\lib\site-
packages\scipy
Python version 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32
bit (Intel)]
nose version 0.10.3
................................................................................
----------------------------------------------------------------------
Ran 233 tests in 2.266s
OK
}}}
constraint Fuzz testing
-----------------------
>python test_distributions_all.py: this is a python script not a nose test
This is how I got started, the file is pretty messy, but it tests all
distributions, continuous distributions with the Kolmogorov-Smirnov test.
For the discrete distributions, it contains a basic comparison of sample
frequencies, cdfs and pmfs. The parameter selection is random, within a
reasonably large set of valid values. There are some parameter ranges,
which are valid, but the distribution tests fail or take a long time to
run; those I have intentionally excluded.
At a not excessively strict level for the tests, the only persistent
failures, I get are hypergeom, and logser (with numpy random numbers).
At higher level of strictness, I get randomly test failures where I
haven't seen any persistent pattern yet.
Besides the kstests, I did not run any tests on the continuous
distribution and their methods.
Since this is my first bugfix attempt in numpy/scipy, I welcome any
feedback.
Josef
--
Ticket URL: <http://scipy.org/scipy/scipy/ticket/745>
SciPy <http://www.scipy.org/>
SciPy is open-source software for mathematics, science, and engineering.
More information about the Scipy-tickets
mailing list