[Numpy-discussion] BigInteger equivalent in numpy

wierob wierob83@googlemail....
Sun Jun 7 04:40:43 CDT 2009


Hi,

int64 and float seem to work for the stderr calculation. Now, the 
calculation of the p-value causes an underflow.

File "C:\Python26\lib\site-packages\scipy\stats\distributions.py", line 2829,in _cdf
    return special.stdtr(df, x)
FloatingPointError: underflow encountered in stdtr

It seems that the error occurs in scipy.special.stdtr(df, x) if df = 
array([13412]) and x = array([61.88071696]).

>>> from scipy import special
>>> import numpy
>>> df = numpy.array([13412])
>>> x = numpy,array([61.88071696])
>>> special.stdtr(df,x)
array([ 1.])
>>> numpy.seterr(all="raise")
>>> special.stdtr(df,x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
FloatingPointError: underflow encountered in stdtr

So, is there another function or datatype that can handle this?

Besides, the overlow in stderr calculation caused nan as result value 
whereas the underflow in p-value calculation eventually leads to 0.0 
(returned by linregress) which is somewhat inconsistent. And in the 
latter case, tricky to identify as an error.

kind regards
robert

josef.pktd@gmail.com schrieb:
> On Thu, Jun 4, 2009 at 8:19 AM, wierob <wierob83@googlemail.com> wrote:
>   
>> Hi,
>>
>> is there a BigInteger equivalent in numpy? The largest integer type I
>> wound was dtype int64.
>>
>> I'm using stats.linregress to perform a regression analysis. The return
>> stderr was nan because stas.ss(...) returned a negative number due to an
>> overflow. Setting dtype to int64 for my input data seems to fix this.
>> But what if my data does not fit in int64?
>>
>> Since Python's long type can hold large data I tried to convert my input
>> to long but it gets converted to int64 in numpy.
>>
>>     
>
> you could try to use floats. stats.ss does the calculation in the same
> type as the input.
> If you convert your input data to floating point you will not get an
> overflow, but floating point precision instead.
>
> Note during the last bugfix, I also changed the implementation of
> stats.linregress and now (0.7.1 and later) it doesn't use stats.ss
> anymore, instead it uses np.cov which always uses floats.
> Also, if you are using an older version there was a mistake in the
> stderr calculations, http://projects.scipy.org/scipy/ticket/874
>
> Josef
>
>   
>> kind regards
>> robert
>>     
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>   


More information about the Numpy-discussion mailing list