[Numpy-discussion] Unexpected float96 precision loss
Thu Sep 2 11:06:23 CDT 2010
Charles R Harris wrote:
>>> So if you write "float96(0.0001)", the result is not the float96 number
>>> closest to 0.0001, but the 96-bit representation of the 64-bit number
>>> closest to 0.0001.
>> but wouldn't it be better to exactly handle strings since those can be
>> converted exactly, which is what the user wants/expects?
> Well, yes. But then we would need to write our own routines for the
The mechanism used in fromstring() and fromfile() (when used for text
parsing) looks like it's designed to support this -- i.e. the actual
string parsing is done be a function provided by the datatype.
I haven't looked to see if there is anything special there for float96.
Does the clib for a compiler that provides a float64 also provide an
atof() function that supports it? Its seems that it should.
Christopher Barker, Ph.D.
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
More information about the NumPy-Discussion