[SciPy-user] fft numerical precision

Ryan Krauss ryanlists at gmail.com
Thu Feb 16 07:41:14 CST 2006


Thanks, I am feeling much better about this.

I actually found something like this on the internet that is really
helpful to me:
In [1]: m=1.0
In [2]: e=m
In [3]: while m+e>m:
   ...:     e=e/2.0
   ...:

In [4]: e
Out[4]: 1.1102230246251565e-16

In [5]: m=1e-4
In [6]: e=m

In [8]: while m+e>m:
   ...:     e=e/2.0
   ...:

In [9]: e
Out[9]: 5.551115123125783e-21

In [10]: m*(2.0**-52)
Out[10]: 2.2204460492503132e-20

Ryan

On 2/15/06, Stephen Walton <stephen.walton at csun.edu> wrote:
> Robert Kern wrote:
>
> >Ryan Krauss wrote:
> >
> >
> >>Sorry to bring this back up again (this thread has been dormant since
> >>12/21/05), but I think this is the key road block between me and
> >>really understanding this problem (and I think it went over my head
> >>when it was originally posted).  If double precision numbers use 52
> >>bits for the mantissa, what does it mean to say that 2**-52 is the
> >>approximate fractional precision of double precision floats?
> >>
> >I don't have time right now to go into a lecture on condition numbers and other
> >such stuff, but let give you an example to chew on and a good reference:
> >
> >
> If I can jump in and show something even simpler:
>
> In [1]: x,sum=0.1,0
>
> In [2]: for m in range(10):
>    ...:     sum = sum + x
>    ...:
>
> In [3]: sum
> Out[3]: 0.99999999999999989
>
> In [4]: sum-1
> Out[4]: -1.1102230246251565e-16
>
> As for books, I've always liked Kahaner, Moler and Nash.
>
> Steve
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.net
> http://www.scipy.net/mailman/listinfo/scipy-user
>



More information about the SciPy-user mailing list