[Scipy-tickets] [SciPy] #745: scipy.stats.distribution - tests and bugfixes

SciPy scipy-tickets@scipy....
Thu Oct 2 08:13:17 CDT 2008


#745: scipy.stats.distribution - tests and bugfixes
-------------------------+--------------------------------------------------
 Reporter:  josefpktd    |       Owner:  somebody
     Type:  defect       |      Status:  new     
 Priority:  normal       |   Milestone:          
Component:  scipy.stats  |     Version:  devel   
 Severity:  normal       |    Keywords:          
-------------------------+--------------------------------------------------
 For original mailing list post see
 http://projects.scipy.org/pipermail/scipy-dev/2008-September/009757.html

 I spend quite some time hunting and killing bugs in scipy.stats and
 learning more about scipy/numpy.

 In the attachment are the scipy.stats.distribution.py file with my
 corrections and comments, two nosetest files and one messy python script
 test file.

 I was working off the trunk at revision 4741, then I merged (with bzr and
 by hand) 2 change sets, 4752 and 4753, into my
 scipy.stats.distributions.py. All current tests and changes are based on
 this file. I did not update any other files, since they did not seem
 directly relevant. (Also, it is the first time that I used bazar for
 merging, but I assume I did it correctly.) NumPy version 1.2.0rc2

 I gave up writing individual bug reports, however, I left many comments in
 the distributions.py file and in some cases one-line testcases, since this
 was the easiest way for me to debug, without having to reload modules all
 the time.


 In terms of importance and sequence, the bug fix categories are roughly:

  * nin correction for vectorize with *args: without this, some methods
 raise exceptions, e.g. _ppf (I think after many shaky earlier attempts, I
 found the correct solution)
  * _ppf did not handle infinite support properly, and requires a .tolist
 in call to `place` (no idea why - casting error in `place`, but this
 works). Also I added a conditional statement to prevent calls when the
 argument list is empty, i.e. [array([]),....].
  * _drv2_moment(self, n, *args) was completely broken, it didn't even have
 a return. It is the base for generic moment calculations.
  * _drv2_ppfsingle: the bisection algorithm had a corner solution wrong
  * def moment: for 2nd moment returned the mean (both discrete and
 continuous distribution

 individual fixes:
  * recipinvgauss_gen: rvs added, didn't work before
  * randint: -1 was missing in _ppf, definitions differ from those in
 description pdf
  * dlaplace: _ppf gave wrong results in tests, correction by Per.Brodtkorb
 in mailing list, but the generic _ppf works for this case also

 * small enhancement proposal (not implemented): include missing cases in
 the support of the function in cdf, ppf, ...

 The above is all for discrete distribution, for the continuous
 distributions, I also did the nin correction now moved, I think correctly,
 to the __init__ method.


 == Test of Basic Properties ==


 The first test just tests for basic properties and their consistency, e.g.
 cdf, pmf, ppf, stats and moments and the internal methods e.g. _ppf. The
 tests check quite a bit for internal consistency (private methods) since I
 used the tests for debugging and trying out fixes. I also added some basic
 tests for survival and inverse survival functions which caught an error in
 the ppf bisection algorithm. The test functions are now split into public
 and private methods.

 {{{
 >python C:\Programs\Python25\Scripts\nosetests-script.py  -s
 test_discrete_basic.py
 ...
 }}}

 with current trunk (rev4741), I get for this test (approximately since
 random):
 {{{
 (copied from email- did not rerun)
 Ran 112 tests in 0.140s
 FAILED (errors=41, failures=15)
 }}}

 with scipy 0.6.0 I get:
 {{{
 Ran 132 tests in 0.797s
 FAILED (errors=65, failures=13)
 }}}

 after the bugfixes, I get:
 {{{
 Ran 132 tests in 5.484s
 FAILED (failures=3)
 }}}

 remaining failures:
  * dlaplace has an error in the sample variance comparison
  * dlaplace and zipf have a error in the generic_moment test which as
 Per.Brodtkorb pointed out should be a private methods since in some cases
 the limits of the support distribution are (by design) not correctly
 initialized yet.


 == Chisquare Test ==


 The second test is a chisquare test for the random variables to be close
 to the theoretical distribution as defined by the .cdf method.

 with current trunk I get for this test:

 {{{
 >python C:\Programs\Python25\Scripts\nosetests-script.py  -s
 test_ndiscrete.py
 or python test_ndiscrete.py

 Ran 12 tests in 0.453s
 FAILED (errors=5)
 }}}

 after patching scipy.stats.distribution the only test failure left are
 in ``logser`` and ``hypergeom`` (for some parameters), where I think the
 numpy.random numbers are wrong



 == scipy.stats tests: no failures ==


 running the current trunk test for scipy.stats on the pre- and post-patch
 distributions.py gives no errors:

 {{{
 >>> from scipy import stats
 >>> stats.test()
 Running unit tests for scipy.stats
 NumPy version 1.2.0rc2
 NumPy is installed in C:\Programs\Python25\lib\site-packages\numpySciPy
 version 0.7.0.dev
 SciPy is installed in C:\Josef\_progs\virtualpy25\envscipy\lib\site-
 packages\scipy
 Python version 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32
 bit (Intel)]
 nose version 0.10.3
 ................................................................................
 ----------------------------------------------------------------------
 Ran 233 tests in 2.266s
 OK
 }}}

 constraint Fuzz testing
 -----------------------
 >python test_distributions_all.py: this is a python script not a nose test

 This is how I got started, the file is pretty messy, but it tests all
 distributions, continuous distributions with the Kolmogorov-Smirnov test.
 For the discrete distributions, it contains a basic comparison of sample
 frequencies, cdfs and pmfs. The parameter selection is random, within a
 reasonably large set of valid values. There are some parameter ranges,
 which are valid, but the distribution tests fail or take a long time to
 run; those I have intentionally excluded.

 At a not excessively strict level for the tests, the only persistent
 failures, I get are hypergeom, and logser (with numpy random numbers).
 At higher level of strictness, I get randomly test failures where I
 haven't seen any persistent pattern yet.

 Besides the kstests, I did not run any tests on the continuous
 distribution and their methods.

 Since this is my first bugfix attempt in numpy/scipy, I welcome any
 feedback.


 Josef

-- 
Ticket URL: <http://scipy.org/scipy/scipy/ticket/745>
SciPy <http://www.scipy.org/>
SciPy is open-source software for mathematics, science, and engineering.


More information about the Scipy-tickets mailing list