[Numpy-discussion] Questions about the array interface.
strawman at astraw.com
Sat Apr 9 12:36:03 CDT 2005
Here's an email Todd Miller sent me (I hoped he'd send it directly to
the list, but I'll forward it. Todd, I hope you don't mind.)
Todd Miller wrote:
> On Fri, 2005-04-08 at 15:46 -0700, Andrew Straw wrote:
>> Hi Todd,
>> Could you join in on this thread? I think you wrote the ieeespecial
>> stuff in numarray, so it's clear you have a much better understanding
>> the issues than I do...
> My own understanding is limited, but I can say a few things that might
> make the status of numarray clearer. My assumptions for numarray were
> 1. Floating point values are 32-bit or 64-bit entities which are stored
> in IEEE-754 format. This is a basic assumption of numarray.ieeespecial
> so I expect it simply won't work on a VAX. There's no checking for
> 2. The platforms that I care about, AMD/Intel Windows/Linux, PowerPC
> OS-X, and Ultra-SPARC Solaris, all seem to provide IEEE-754 floating
> point. ieeespecial has been tested to work there.
> 3. I viewed IEEE-754 floating point numbers as 32-bit or 64-bit
> integers, and contiguous ranges on those integers are used to
> special values like NAN and INF. Platform byte ordering for the
> IEEE-754 floating point numbers mirrors byte ordering for integers so
> the ieeespecial NAN detection code works in a cross platform way *and*
> values exported from one IEEE-754 platform will work with ieeespecial
> when imported on another. It's important to note that special values
> are not unique: there is no single NAN value; it's a bit range.
> 4. numarray leaks IEEE-754 special values out into Python floating
> scalars. This may be bad form. I do this because (1) they repr
> understandably if not in a platform independent way and (2) people need
> to get at them. I noticed recently that ieeespecial.nan ==
> ieeespecial.nan returns incorrect answers (True!) for Python-2.3 and
> correct ones (False) for Python-2.4. I haven't looked at what the
> version does yet: array(nan) == array(nan). The point to be taken
> this is that the level at which numarray ieee special value handling
> works or doesn't work is really restricted to (1) detecting certain
> ieee-754 bit ranges (2) the basic behavior of C code for C89 complilers
> for array code (no guarantees) (3) the behavior of Python itself
> In the context of the array protocol (looking very nice by the way) my
> thinking is that non-IEEE-754 floating point could be described with
> fields and that the current type codes should mean IEEE-754.
> Some minor things I noticed in the array interface:
> 1. The packing order of bit fields is not clear. In C, my experience
> is that some compilers pack bit structs towards the higher order bits
> an integer, and some towards the lower. More info to clarify that
> would be helpful.
> 2. I saw no mention that we're talking about a protocol. I'm sure
> that's clear to everyone following this discussion closely, but I
> didn't see it in the spec. It might make sense to allude to the C
> helper functions and potential for additions to the Python type struct
> even if they're not spelled out.
On Apr 9, 2005, at 9:54 AM, Travis Oliphant wrote:
> konrad.hinsen at laposte.net wrote:
>> On 09.04.2005, at 01:04, Scott Gilbert wrote:
>>> I think something we've been assuming is that the array data is
>>> IEEE-754 compliant (maybe it needs to be byteswapped). If that's
>>> not true,
>>> then we're going to need some new typecodes. We're not supporting
>>> ability to pass VAX floating point around (Are we????).
> No, in moving from the struct modules character codes we are trying to
> do something more platform independent because it is very likely that
> different platforms will want to exchange binary data. IEEE-754 is a
> great standard to build an interface around. Data sharing was the
> whole reason the standard emerged and a lot of companies got on board.
>> This discussion has been coming up regularly for a few years. Until
>> now the concensus has always been that Python should make no
>> assumptions that go beyond what a C compiler can promise. Which
>> means no assumptions about floating-point representation.
>> Of course the computing world is changing, and IEEE format may well
>> be ubiquitous by now. Vaxes must be in the museum by now. But how
>> about mainframes? IBM mainframes didn't use IEEE when I used them
>> (last time 15 years ago), and they are still around, possibly
>> compatible to their ancestors.
> I found the following piece, written about 6 years ago interesting:
> Basically, it states that chips in newer IBM mainframes support the
> IEEE 754 standard.
>> Another detail to consider is that although most machines use the
>> IEEE representation, hardly any respects the IEEE rules for floating
>> point operations in all detail. In particular, trusting that Inf and
>> NaN will be treated as IEEE postulates is a risky business.
> But, this can be handled with platform-dependendent C-code when and if
> problems arise.
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real
> Discover which products truly live up to the hype. Start reading now.
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
More information about the Numpy-discussion