[SciPy-user] fft numerical precision

Stephen Walton stephen.walton at csun.edu
Wed Feb 15 22:57:37 CST 2006


Robert Kern wrote:

>Ryan Krauss wrote:
>  
>
>>Sorry to bring this back up again (this thread has been dormant since
>>12/21/05), but I think this is the key road block between me and
>>really understanding this problem (and I think it went over my head
>>when it was originally posted).  If double precision numbers use 52
>>bits for the mantissa, what does it mean to say that 2**-52 is the
>>approximate fractional precision of double precision floats? 
>>
>I don't have time right now to go into a lecture on condition numbers and other
>such stuff, but let give you an example to chew on and a good reference:
>  
>
If I can jump in and show something even simpler:

In [1]: x,sum=0.1,0

In [2]: for m in range(10):
   ...:     sum = sum + x
   ...:

In [3]: sum
Out[3]: 0.99999999999999989

In [4]: sum-1
Out[4]: -1.1102230246251565e-16

As for books, I've always liked Kahaner, Moler and Nash.

Steve



More information about the SciPy-user mailing list