[Numpy-discussion] float128 casting rounding as if it were float64

Matthew Brett matthew.brett@gmail....
Tue Oct 11 13:17:47 CDT 2011


While struggling with floating point precision, I ran into this:

In [52]: a = 2**54+3

In [53]: a
Out[53]: 18014398509481987L

In [54]: np.float128(a)
Out[54]: 18014398509481988.0

In [55]: np.float128(a)-1
Out[55]: 18014398509481987.0

The line above tells us that float128 can exactly represent 2**54+3,
but the line above that says that np.float128(2**54+3) rounds upwards
as if it were a float64:

In [59]: np.float64(a)
Out[59]: 18014398509481988.0

In [60]: np.float64(a)-1
Out[60]: 18014398509481988.0


In [66]: np.float128('1e308')
Out[66]: 1.000000000000000011e+308

In [67]: np.float128('1e309')
Out[67]: inf

Is it possible that float64 is being used somewhere in float128 casting?



More information about the NumPy-Discussion mailing list