[Numpy-discussion] Unexpected float96 precision loss
Charles R Harris
Wed Sep 1 20:03:00 CDT 2010
On Wed, Sep 1, 2010 at 3:30 PM, Michael Gilbert <email@example.com
> On Wed, 1 Sep 2010 21:15:22 +0000 (UTC), Pauli Virtanen wrote:
> > Wed, 01 Sep 2010 16:26:59 -0400, Michael Gilbert wrote:
> > > I've been using numpy's float96 class lately, and I've run into some
> > > strange precision errors.
> > [clip]
> > > >>> x = numpy.array( [0.01] , numpy.float96 )
> > [clip]
> > > I would expect the float96 calculation to also produce 0.0 exactly as
> > > found in the float32 and float64 examples. Why isn't this the case?
> > (i) It is not possible to write long double literals in Python.
> > "float96(0.0001)" means in fact "float96(float64(0.0001))"
> > (ii) It is not possible to represent numbers 10^-r, r > 1 exactly
> > in base-2 floating point.
> > So if you write "float96(0.0001)", the result is not the float96 number
> > closest to 0.0001, but the 96-bit representation of the 64-bit number
> > closest to 0.0001. Indeed,
> > >>> float96(0.0001), float96(1.0)/1000
> > (0.00010000000000000000479, 0.00099999999999999999996)
> Interesting. float96( '0.0001' ) also seems to evaluate to the first
> result. I assume that it also does a float64( '0.0001' ) conversion
> first. I understand that you can't change how python passes in floats,
> but wouldn't it be better to exactly handle strings since those can be
> converted exactly, which is what the user wants/expects?
Well, yes. But then we would need to write our own routines for the
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the NumPy-Discussion