[SciPy-User] Single or double precision?
Mon Oct 29 09:21:26 CDT 2012
On 10/29/12 9:18 AM, Sturla Molden wrote:
> Sorry this might be a bit off-topic. But I'll ask on this list anyway:
> Given that a signal is sampled at 16 bits resolution (at an ADC), will
> it always be sufficient to store a filtered version at single precision?
> I.e. a float32 has a 23 bit mantissa, so the truncation error should be
> tiny compared to the digitization error from the 16 bits ADC. Or am I
> thinking wrongly about this? Usually I don't care and just use double
> precision everywhere. But I will save gigabytes of store space by using
> single precision here.
In : a = np.arange(2**16, dtype=np.uint16)
In : np.all(a.astype(np.float32).astype(np.uint16) == a)
So I think you will be safe here.
More information about the SciPy-User