[SciPy-user] fft numerical precision
sransom at nrao.edu
Wed Dec 21 13:58:23 CST 2005
Note that 1e-16 is the approximate _fractional_ precision of double
precision floats. You can get absolute numbers much smaller than that.
There are a bunch of good references to floating point accuracy etc.
Some of them are linked here:
Note that the FFTW website lists FFT accuracies in both single and
double precision here:
On Wednesday 21 December 2005 02:53 pm, Ryan Krauss wrote:
> I have run into this as well. 1e-16 seems large to me for double
> precision. Robert can you explain this a bit more please. Is there
> an IEEE spec or something that specifies how much of the 64bits is
> for exponent and how much is for the mantissa (I think that is the
> right word)? I was playing with some FORTRAN code and it seems like
> there was a big difference with complex vs. double complex. It seems
> like 1e-16 was the magnitude floor of complex and double complex was
> On 12/21/05, Robert Kern <robert.kern at gmail.com> wrote:
> > Darren Dale wrote:
> > > I am running into problems related, I think, to numerical
> > > precision of fast Fourier transforms. If I Fourier transform a
> > > gaussian distribution:
> > >
> > > absolute(fft(stats.norm.pdf(linspace(-10,10,512), loc=0,
> > > scale=1)))
> > >
> > > I find a floor of about 1e-16. Does anyone know of a way to
> > > improve the precision?
> > 1e-16 is the best you can do with double precision.
> > --
> > Robert Kern
> > robert.kern at gmail.com
> > "In the fields of hell where the grass grows high
> > Are the graves of dreams allowed to die."
> > -- Richard Harter
> > _______________________________________________
> > SciPy-user mailing list
> > SciPy-user at scipy.net
> > http://www.scipy.net/mailman/listinfo/scipy-user
> SciPy-user mailing list
> SciPy-user at scipy.net
Scott M. Ransom Address: NRAO
Phone: (434) 296-0320 520 Edgemont Rd.
email: sransom at nrao.edu Charlottesville, VA 22903 USA
GPG Fingerprint: 06A9 9553 78BE 16DB 407B FFCA 9BFA B6FF FFD3 2989
More information about the SciPy-user