[SciPy-Dev] cephes_smirnov never returns on mips/sparc/...

[josef.pktd@gmai...]

On Fri, Mar 30, 2012 at 12:33 PM,  <josef.pktd@gmail.com> wrote:
> On Fri, Mar 30, 2012 at 11:43 AM, Yaroslav Halchenko
> <lists@onerussian.com> wrote:
>> I have reported this issue some time ago on Debian
>> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=653948
>> and then forgot about it until now that I ran into it again.
>>
>> With recent scipy (including git master 0fbfdbc)
>>
>> scipy.stats.ksone.fit
>>
>> seems to stall (never return) on big-endian boxes and return (1.0, nan,
>> nan) on x86.  It seems that it is  working correctly with scipy
>> 0.7.2 (as it is now in Debian stable).
>>
>> So -- is that anything known?
>
> No, never seen before.
>
> I didn't think anyone would ever fit ksone, It doesn't even have a pdf
> defined in the source. It's mainly included for Kolmogorov-Smirnov
> test.
>
>
> I don't know how to interpret gdp. Is it clear that it is
> cephes_smirnov? Do you know the values that trigger it?
> If not, you could put a print in the ksone_gen._cdf to see where it
> gets stuck. (it might print a lot)
>
> I think it's a bug for scipy.stats that ksone doesn't define the
> support boundary .b, it looks like it should be (0,1) (default a=0
> looks ok.)
> There will be lots of nans during fit().  I don't think the generic
> fit is smart enough to figure out non-nan or non-inf starting values
> for your dataset. (There is a ticket to avoid inf in starting values
> but it hasn't been included yet.)

Even with reasonable (?) starting values, it looks like there are
problems with maximum likelihood for ksone, local maxima?

>>> ss.ksone.b=1
>>> ss.ksone.fit(d, 1, loc=d.min()-0.1, scale=(d.max()-d.min()+0.1)*2)
(1.3013055706786489, -0.35244788860684106, 0.7889942595986128)
>>> ss.ksone.fit(d, 1, loc=d.min()-0.1, scale=(d.max()-d.min()+0.1)*3)
(1.9999958481711952, -0.4279954520552498, 0.86455690493638671)
>>> ss.ksone.fit(d, 1, loc=d.min()-0.1, scale=(d.max()-d.min()+0.1)*4)
(1.2928641767257472, -0.35244871300537894, 0.788994096846098)
>>> ss.ksone.fit(d, 1, loc=d.min()-0.1, scale=(d.max()-d.min()+0.1)*5)
(1.719273457432017, -0.3524476012033948, 0.7889933454348288)
>>> ss.ksone.fit(d, 1, loc=d.min()-0.2, scale=(d.max()-d.min()+0.1)*5)
(1.9999992822317982, -0.54142710315126585, 0.97805283203179272)
>>> ss.ksone.fit(d, 1, loc=d.min()-0.3, scale=(d.max()-d.min()+0.1)*5)
(1.0000030742081174, -0.65522050691419065, 1.0917721006638095)
>>> ss.ksone.fit(d, 10, loc=d.min()-0.3, scale=(d.max()-d.min()+0.1)*5)
(15.292790569198694, -0.35248279275767735, 2.5098888045388446)

Josef

>
> Josef
>
>>
>> snippet to replicate:
>>
>> import numpy as np
>> import scipy.stats as ss
>>
>>
>> d = np.array([-0.18879233,  0.15734249,  0.18695107,  0.27908787, -0.248649,
>>              -0.2171497 ,  0.12233512,  0.15126419,  0.03119282,  0.4365294 ,
>>              0.08930393, -0.23509903,  0.28231224, -0.09974875, -0.25196048,
>>              0.11102028,  0.1427649 ,  0.10176452,  0.18754054,  0.25826724,
>>              0.05988819,  0.0531668 ,  0.21906056,  0.32106729,  0.2117662 ,
>>              0.10886442,  0.09375789,  0.24583286, -0.22968366, -0.07842391,
>>              -0.31195432, -0.21271196,  0.1114243 , -0.13293002,  0.01331725,
>>              -0.04330977, -0.09485776, -0.28434547,  0.22245721, -0.18518199,
>>              -0.10943985, -0.35243174,  0.06897665, -0.03553363, -0.0701746 ,
>>              -0.06037974,  0.37670779, -0.21684405])
>>
>> print "Fitting now"
>> print ss.ksone.fit(d)
>>
>>
>> --
>> =------------------------------------------------------------------=
>> Keep in touch                                     www.onerussian.com
>> Yaroslav Halchenko                 www.ohloh.net/accounts/yarikoptic
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev@scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev