[SciPy-User] Single or double precision?

Francesc Alted francesc@continuum...
Mon Oct 29 09:21:26 CDT 2012


On 10/29/12 9:18 AM, Sturla Molden wrote:
> Sorry this might be a bit off-topic. But I'll ask on this list anyway:
>
> Given that a signal is sampled at 16 bits resolution (at an ADC), will
> it always be sufficient to store a filtered version at single precision?
> I.e. a float32 has a 23 bit mantissa, so the truncation error should be
> tiny compared to the digitization error from the 16 bits ADC. Or am I
> thinking wrongly about this? Usually I don't care and just use double
> precision everywhere. But I will save gigabytes of store space by using
> single precision here.

In []: a = np.arange(2**16, dtype=np.uint16)

In []: np.all(a.astype(np.float32).astype(np.uint16) == a)
Out[]: True


So I think you will be safe here.

-- 
Francesc Alted



More information about the SciPy-User mailing list