[Scipy-tickets] [SciPy] #1546: stats: test_continuous_basic has broken test

SciPy Trac scipy-tickets@scipy....
Sun Oct 30 16:15:29 CDT 2011


#1546: stats: test_continuous_basic has broken test
-------------------------+--------------------------------------------------
 Reporter:  josefpktd    |       Owner:  somebody   
     Type:  defect       |      Status:  new        
 Priority:  normal       |   Milestone:  Unscheduled
Component:  scipy.stats  |     Version:  0.9.0      
 Keywords:               |  
-------------------------+--------------------------------------------------

Comment(by josefpktd):

 Doing the obvious correction of the typo still didn't catch tukeylambda.
 Finally, I realized that we need two sided tests, even though it was
 mentioned in the docstring. (It's not like tests of nested models that
 only need to check one side.)

 my current version, edited on top of scipy 0.9, is

 {{{
 def check_sample_var(sv,n, popvar):
     '''
 two-sided chisquare test for sample variance equal to hypothesized
 variance
     '''

     df = n-1
     chi2 = (n-1)*sv/float(popvar)
     #pval = stats.chisqprob(chi2,df)*2
     pval = min(stats.chi2.cdf(chi2,df), stats.chi2.sf(chi2,df)) #need two-
 sided
     npt.assert_(pval > 0.005, 'var fail, t,pval = %f, %f, v,sv=%f,%f' %
 (chi2,pval,popvar,sv))
 }}}


 and has with the very low pvalue 0.005 the following failures (there are
 also 12 failures at pval=0.01) with the not-slow option

 {{{
 ======================================================================
 FAIL:
 test_continuous_basic.test_cont_basic(<scipy.stats.distributions.fatiguelife_gen
 object at 0x02F118F0>, (29,), array(421.5), array(884942.25),
 381.5231692182964, 659709.81429483229, 1000, 'fatiguelifesample mean
 test')
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "c:\python26\lib\site-packages\nose-1.0.0-py2.6.egg\nose\case.py",
 line 187, in runTest
     self.test(*self.arg)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 164, in wrap
     return func(*a, **kw)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 259, in
 check_sample_meanvar_
     check_sample_var(sv, sn, v)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 292, in
 check_sample_var
     npt.assert_(pval > 0.005, 'var fail, t,pval = %f, %f, v,sv=%f,%f' %
 (chi2,pval,popvar,sv))
   File "C:\Python26\lib\site-packages\numpy\testing\utils.py", line 34, in
 assert_
     raise AssertionError(msg)
 AssertionError: var fail, t,pval = 744.737981, 0.000000,
 v,sv=884942.250000,659709.814295

 ======================================================================
 FAIL:
 test_continuous_basic.test_cont_basic(<scipy.stats.distributions.gamma_gen
 object at 0x02F1A130>, (1.9932305483800778,), array(1.9932305483800778),
 array(1.9932305483800778), 1.9049475453784119, 1.683287232178736, 1000,
 'gammasample mean test')
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "c:\python26\lib\site-packages\nose-1.0.0-py2.6.egg\nose\case.py",
 line 187, in runTest
     self.test(*self.arg)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 164, in wrap
     return func(*a, **kw)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 259, in
 check_sample_meanvar_
     check_sample_var(sv, sn, v)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 292, in
 check_sample_var
     npt.assert_(pval > 0.005, 'var fail, t,pval = %f, %f, v,sv=%f,%f' %
 (chi2,pval,popvar,sv))
   File "C:\Python26\lib\site-packages\numpy\testing\utils.py", line 34, in
 assert_
     raise AssertionError(msg)
 AssertionError: var fail, t,pval = 843.657522, 0.000127,
 v,sv=1.993231,1.683287

 ======================================================================
 FAIL:
 test_continuous_basic.test_cont_basic(<scipy.stats.distributions.genlogistic_gen
 object at 0x02F11EB0>, (0.41192440799679475,), array(-1.8996417990533176),
 array(8.5520107632574316), -1.6915023776411073, 7.2701839656308955, 1000,
 'genlogisticsample mean test')
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "c:\python26\lib\site-packages\nose-1.0.0-py2.6.egg\nose\case.py",
 line 187, in runTest
     self.test(*self.arg)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 164, in wrap
     return func(*a, **kw)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 259, in
 check_sample_meanvar_
     check_sample_var(sv, sn, v)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 292, in
 check_sample_var
     npt.assert_(pval > 0.005, 'var fail, t,pval = %f, %f, v,sv=%f,%f' %
 (chi2,pval,popvar,sv))
   File "C:\Python26\lib\site-packages\numpy\testing\utils.py", line 34, in
 assert_
     raise AssertionError(msg)
 AssertionError: var fail, t,pval = 849.263873, 0.000217,
 v,sv=8.552011,7.270184

 ======================================================================
 FAIL:
 test_continuous_basic.test_cont_basic(<scipy.stats.distributions.gilbrat_gen
 object at 0x02F234B0>, (), array(1.6487212707001282),
 array(4.670774270471604), 1.5542857947354762, 3.0177319205419764, 1000,
 'gilbratsample mean test')
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "c:\python26\lib\site-packages\nose-1.0.0-py2.6.egg\nose\case.py",
 line 187, in runTest
     self.test(*self.arg)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 164, in wrap
     return func(*a, **kw)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 259, in
 check_sample_meanvar_
     check_sample_var(sv, sn, v)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 292, in
 check_sample_var
     npt.assert_(pval > 0.005, 'var fail, t,pval = %f, %f, v,sv=%f,%f' %
 (chi2,pval,popvar,sv))
   File "C:\Python26\lib\site-packages\numpy\testing\utils.py", line 34, in
 assert_
     raise AssertionError(msg)
 AssertionError: var fail, t,pval = 645.442065, 0.000000,
 v,sv=4.670774,3.017732

 ======================================================================
 FAIL:
 test_continuous_basic.test_cont_basic(<scipy.stats.distributions.gumbel_l_gen
 object at 0x02F1A590>, (), array(-0.57721566490153287),
 array(1.6449340668482264), -0.48770832402349951, 1.403145247583542, 1000,
 'gumbel_lsample mean test')
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "c:\python26\lib\site-packages\nose-1.0.0-py2.6.egg\nose\case.py",
 line 187, in runTest
     self.test(*self.arg)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 164, in wrap
     return func(*a, **kw)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 259, in
 check_sample_meanvar_
     check_sample_var(sv, sn, v)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 292, in
 check_sample_var
     npt.assert_(pval > 0.005, 'var fail, t,pval = %f, %f, v,sv=%f,%f' %
 (chi2,pval,popvar,sv))
   File "C:\Python26\lib\site-packages\numpy\testing\utils.py", line 34, in
 assert_
     raise AssertionError(msg)
 AssertionError: var fail, t,pval = 852.157014, 0.000284,
 v,sv=1.644934,1.403145

 ======================================================================
 FAIL:
 test_continuous_basic.test_cont_basic(<scipy.stats.distributions.invgamma_gen
 object at 0x02F1A9B0>, (2.0668996136993067,), array(0.93729530609975309),
 array(13.131951625091878), 0.98003361245261933, 2.2163927217803887, 1000,
 'invgammasample mean test')
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "c:\python26\lib\site-packages\nose-1.0.0-py2.6.egg\nose\case.py",
 line 187, in runTest
     self.test(*self.arg)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 164, in wrap
     return func(*a, **kw)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 259, in
 check_sample_meanvar_
     check_sample_var(sv, sn, v)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 292, in
 check_sample_var
     npt.assert_(pval > 0.005, 'var fail, t,pval = %f, %f, v,sv=%f,%f' %
 (chi2,pval,popvar,sv))
   File "C:\Python26\lib\site-packages\numpy\testing\utils.py", line 34, in
 assert_
     raise AssertionError(msg)
 AssertionError: var fail, t,pval = 168.609845, 0.000000,
 v,sv=13.131952,2.216393

 ======================================================================
 FAIL:
 test_continuous_basic.test_cont_basic(<scipy.stats.distributions.loggamma_gen
 object at 0x02F23270>, (0.41411931826052117,), array(-2.4617679388246936),
 array(6.8426502245961007), -2.4057296757832174, 5.9831621683265688, 1000,
 'loggammasample mean test')
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "c:\python26\lib\site-packages\nose-1.0.0-py2.6.egg\nose\case.py",
 line 187, in runTest
     self.test(*self.arg)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 164, in wrap
     return func(*a, **kw)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 259, in
 check_sample_meanvar_
     check_sample_var(sv, sn, v)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 292, in
 check_sample_var
     npt.assert_(pval > 0.005, 'var fail, t,pval = %f, %f, v,sv=%f,%f' %
 (chi2,pval,popvar,sv))
   File "C:\Python26\lib\site-packages\numpy\testing\utils.py", line 34, in
 assert_
     raise AssertionError(msg)
 AssertionError: var fail, t,pval = 873.518127, 0.001755,
 v,sv=6.842650,5.983162

 ======================================================================
 FAIL:
 test_continuous_basic.test_cont_basic(<scipy.stats.distributions.lognorm_gen
 object at 0x02F233F0>, (0.95368226960575331,), array(1.5757871665018548),
 array(3.6827062109137709), 1.4935980843433927, 2.4720855647743383, 1000,
 'lognormsample mean test')
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "c:\python26\lib\site-packages\nose-1.0.0-py2.6.egg\nose\case.py",
 line 187, in runTest
     self.test(*self.arg)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 164, in wrap
     return func(*a, **kw)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 259, in
 check_sample_meanvar_
     check_sample_var(sv, sn, v)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 292, in
 check_sample_var
     npt.assert_(pval > 0.005, 'var fail, t,pval = %f, %f, v,sv=%f,%f' %
 (chi2,pval,popvar,sv))
   File "C:\Python26\lib\site-packages\numpy\testing\utils.py", line 34, in
 assert_
     raise AssertionError(msg)
 AssertionError: var fail, t,pval = 670.597473, 0.000000,
 v,sv=3.682706,2.472086

 ======================================================================
 FAIL:
 test_continuous_basic.test_cont_basic(<scipy.stats.distributions.pareto_gen
 object at 0x02F23AD0>, (2.621716532144454,), array(1.6166305764162521),
 array(1.6034057205287133), 1.6550137301062546, 1.2674974417903764, 1000,
 'paretosample mean test')
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "c:\python26\lib\site-packages\nose-1.0.0-py2.6.egg\nose\case.py",
 line 187, in runTest
     self.test(*self.arg)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 164, in wrap
     return func(*a, **kw)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 259, in
 check_sample_meanvar_
     check_sample_var(sv, sn, v)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 292, in
 check_sample_var
     npt.assert_(pval > 0.005, 'var fail, t,pval = %f, %f, v,sv=%f,%f' %
 (chi2,pval,popvar,sv))
   File "C:\Python26\lib\site-packages\numpy\testing\utils.py", line 34, in
 assert_
     raise AssertionError(msg)
 AssertionError: var fail, t,pval = 789.712752, 0.000000,
 v,sv=1.603406,1.267497

 ======================================================================
 FAIL:
 test_continuous_basic.test_cont_basic(<scipy.stats.distributions.powerlaw_gen
 object at 0x02F23C70>, (1.6591133289905851,), array(0.62393479469353574),
 array(0.85857496135780687), 0.63812341708244613, 0.061503031152340903,
 1000, 'powerlawsample mean test')
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "c:\python26\lib\site-packages\nose-1.0.0-py2.6.egg\nose\case.py",
 line 187, in runTest
     self.test(*self.arg)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 164, in wrap
     return func(*a, **kw)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 259, in
 check_sample_meanvar_
     check_sample_var(sv, sn, v)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 292, in
 check_sample_var
     npt.assert_(pval > 0.005, 'var fail, t,pval = %f, %f, v,sv=%f,%f' %
 (chi2,pval,popvar,sv))
   File "C:\Python26\lib\site-packages\numpy\testing\utils.py", line 34, in
 assert_
     raise AssertionError(msg)
 AssertionError: var fail, t,pval = 71.562217, 0.000000,
 v,sv=0.858575,0.061503

 ======================================================================
 FAIL:
 test_continuous_basic.test_cont_basic(<scipy.stats.distributions.tukeylambda_gen
 object at 0x02FD6590>, (3.1321477856738267,), array(0.0),
 array(0.30476472279111871), 0.0090800061342219563, 0.026536393118696645,
 1000, 'tukeylambdasample mean test')
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "c:\python26\lib\site-packages\nose-1.0.0-py2.6.egg\nose\case.py",
 line 187, in runTest
     self.test(*self.arg)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 164, in wrap
     return func(*a, **kw)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 259, in
 check_sample_meanvar_
     check_sample_var(sv, sn, v)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 292, in
 check_sample_var
     npt.assert_(pval > 0.005, 'var fail, t,pval = %f, %f, v,sv=%f,%f' %
 (chi2,pval,popvar,sv))
   File "C:\Python26\lib\site-packages\numpy\testing\utils.py", line 34, in
 assert_
     raise AssertionError(msg)
 AssertionError: var fail, t,pval = 86.984663, 0.000000,
 v,sv=0.304765,0.026536

 ======================================================================
 FAIL:
 test_continuous_basic.test_cont_basic(<scipy.stats.distributions.tukeylambda_gen
 object at 0x02FD6590>, (0.14000000000000001,), array(0.0),
 array(24.379664446968761), 0.090612433601054559, 2.0217342128370261, 1000,
 'tukeylambdasample mean test')
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "c:\python26\lib\site-packages\nose-1.0.0-py2.6.egg\nose\case.py",
 line 187, in runTest
     self.test(*self.arg)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 164, in wrap
     return func(*a, **kw)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 259, in
 check_sample_meanvar_
     check_sample_var(sv, sn, v)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 292, in
 check_sample_var
     npt.assert_(pval > 0.005, 'var fail, t,pval = %f, %f, v,sv=%f,%f' %
 (chi2,pval,popvar,sv))
   File "C:\Python26\lib\site-packages\numpy\testing\utils.py", line 34, in
 assert_
     raise AssertionError(msg)
 AssertionError: var fail, t,pval = 82.844146, 0.000000,
 v,sv=24.379664,2.021734

 ----------------------------------------------------------------------
 Ran 1149 tests in 10.499s

 FAILED (SKIP=1, failures=12)
 }}}

 3 more failures in the "slow" distributions

 {{{
 ======================================================================
 FAIL:
 test_continuous_basic.test_cont_basic_slow(<scipy.stats.distributions.johnsonsu_gen
 object at 0x02DFFE10>, (2.554395574161155, 2.2482281679651965),
 array(-1.5421538673934436), array(0.76298819698253784),
 -1.4802090580087397, 0.64030549378085866, 1000, 'johnsonsusample mean
 test')
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "c:\python26\lib\site-packages\nose-1.0.0-py2.6.egg\nose\case.py",
 line 187, in runTest
     self.test(*self.arg)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 164, in wrap
     return func(*a, **kw)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 259, in
 check_sample_meanvar_
     check_sample_var(sv, sn, v)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 292, in
 check_sample_var
     npt.assert_(pval > 0.01, 'var fail, t,pval = %f, %f, v,sv=%f,%f' %
 (chi2,pval,popvar,sv))
   File "C:\Python26\lib\site-packages\numpy\testing\utils.py", line 34, in
 assert_
     raise AssertionError(msg)
 AssertionError: var fail, t,pval = 838.368393, 0.000075,
 v,sv=0.762988,0.640305

 ======================================================================
 FAIL:
 test_continuous_basic.test_cont_basic_slow(<scipy.stats.distributions.rdist_gen
 object at 0x02E07ED0>, (0.90000000000000002,), array(0.0),
 array(1.7800284850122077), 0.034195658437930029, 0.53053935902899019,
 1000, 'rdistsample mean test')
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "c:\python26\lib\site-packages\nose-1.0.0-py2.6.egg\nose\case.py",
 line 187, in runTest
     self.test(*self.arg)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 164, in wrap
     return func(*a, **kw)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 259, in
 check_sample_meanvar_
     check_sample_var(sv, sn, v)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 292, in
 check_sample_var
     npt.assert_(pval > 0.01, 'var fail, t,pval = %f, %f, v,sv=%f,%f' %
 (chi2,pval,popvar,sv))
   File "C:\Python26\lib\site-packages\numpy\testing\utils.py", line 34, in
 assert_
     raise AssertionError(msg)
 AssertionError: var fail, t,pval = 297.752999, 0.000000,
 v,sv=1.780028,0.530539

 ======================================================================
 FAIL:
 test_continuous_basic.test_cont_basic_slow(<scipy.stats.distributions.recipinvgauss_gen
 object
 at 0x02ECD1F0>, (0.63004267809369119,), array(2.5871940660171391),
 array(3.5871940659111603), 2.4895378659294742, 3.0462804278153826, 1000,
 'recipinvgausssample mean test')
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "c:\python26\lib\site-packages\nose-1.0.0-py2.6.egg\nose\case.py",
 line 187, in runTest
     self.test(*self.arg)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 164, in wrap
     return func(*a, **kw)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 259, in
 check_sample_meanvar_
     check_sample_var(sv, sn, v)
   File "C:\Python26\Lib\site-
 packages\scipy\stats\tests\test_continuous_basic.py", line 292, in
 check_sample_var
     npt.assert_(pval > 0.01, 'var fail, t,pval = %f, %f, v,sv=%f,%f' %
 (chi2,pval,popvar,sv))
   File "C:\Python26\lib\site-packages\numpy\testing\utils.py", line 34, in
 assert_
     raise AssertionError(msg)
 AssertionError: var fail, t,pval = 848.360610, 0.000199,
 v,sv=3.587194,3.046280

 ----------------------------------------------------------------------
 Ran 1576 tests in 169.179s

 FAILED (SKIP=1, failures=15)
 }}}

-- 
Ticket URL: <http://projects.scipy.org/scipy/ticket/1546#comment:2>
SciPy <http://www.scipy.org>
SciPy is open-source software for mathematics, science, and engineering.


More information about the Scipy-tickets mailing list