[Numpy-discussion] Questions about the array interface.

Andrew Straw strawman at astraw.com
Sat Apr 9 12:36:03 CDT 2005


Here's an email Todd Miller sent me (I hoped he'd send it directly to 
the list, but I'll forward it.  Todd, I hope you don't mind.)

Todd Miller wrote:

> On Fri, 2005-04-08 at 15:46 -0700, Andrew Straw wrote:
>> Hi Todd,
>>
>> Could you join in on this thread?  I think you wrote the ieeespecial
>> stuff in numarray, so it's clear you have a much better understanding 
>> of
>> the issues than I do...
>>
>> Cheers!
>> Andrew
>
> My own understanding is limited,  but I can say a few things that might
> make the status of numarray clearer.  My assumptions for numarray were
> that:
>
> 1. Floating point values are 32-bit or 64-bit entities which are stored
> in IEEE-754 format.  This is a basic assumption of numarray.ieeespecial
> so I expect it simply won't work on a VAX.  There's no checking for
> this.
>
> 2. The platforms that I care about,  AMD/Intel Windows/Linux, PowerPC
> OS-X, and Ultra-SPARC Solaris,  all seem to provide IEEE-754 floating
> point.  ieeespecial has been tested to work there.
>
> 3. I viewed IEEE-754 floating point numbers as 32-bit or 64-bit 
> unsigned
> integers,  and contiguous ranges on those integers are used to 
> represent
> special values like NAN and INF.  Platform byte ordering for the
> IEEE-754 floating point numbers mirrors byte ordering for integers so
> the ieeespecial NAN detection code works in a cross platform way *and*
> values exported from one IEEE-754 platform will work with ieeespecial
> when imported on another.  It's important to note that special values
> are not unique:  there is no single NAN value;  it's a bit range.
>
> 4. numarray leaks IEEE-754 special values out into Python floating 
> point
> scalars.  This may be bad form.  I do this because (1) they repr
> understandably if not in a platform independent way and (2) people need
> to get at them.  I noticed recently that ieeespecial.nan ==
> ieeespecial.nan returns incorrect answers (True!) for Python-2.3 and
> correct ones (False) for Python-2.4.  I haven't looked at what the 
> array
> version does yet:  array(nan) == array(nan).  The point to be taken 
> from
> this is that the level at which numarray ieee special value handling
> works or doesn't work is really restricted to (1) detecting certain
> ieee-754 bit ranges (2) the basic behavior of C code for C89 complilers
> for array code (no guarantees) (3) the behavior of Python itself
> (improving).
>
> In the context of the array protocol (looking very nice by the way) my
> thinking is that non-IEEE-754 floating point could be described with 
> bit
> fields and that the current type codes should mean IEEE-754.
>
> Some minor things I noticed in the array interface:
>
> 1. The packing order of bit fields is not clear.  In C,  my experience
> is that some compilers pack bit structs towards the higher order bits 
> of
> an integer,  and some towards the lower.  More info to clarify that
> would be helpful.
>
> 2.  I saw no mention that we're talking about a protocol.  I'm sure
> that's clear to everyone following this discussion closely,  but I
> didn't see it in the spec.  It might make sense to allude to the C
> helper functions and potential for additions to the Python type struct
> even if they're not spelled out.
>
> Regards,
> Todd


On Apr 9, 2005, at 9:54 AM, Travis Oliphant wrote:

> konrad.hinsen at laposte.net wrote:
>
>> On 09.04.2005, at 01:04, Scott Gilbert wrote:
>>
>>> I think something we've been assuming is that the array data is  
>>> basically
>>> IEEE-754 compliant (maybe it needs to be byteswapped).  If that's 
>>> not  true,
>>> then we're going to need some new typecodes.  We're not supporting 
>>> the
>>> ability to pass VAX floating point around (Are we????).
>>
>
> No, in moving from the struct modules character codes we are trying to 
> do something more platform independent because it is very likely that 
> different platforms will want to exchange binary data.   IEEE-754 is a 
> great standard to build an interface around.   Data sharing was the 
> whole reason the standard emerged and a lot of companies got on board.
>
>>
>> This discussion has been coming up regularly for a few years. Until 
>> now  the concensus has always been that Python should make no 
>> assumptions  that go beyond what a C compiler can promise. Which 
>> means no  assumptions about floating-point representation.
>>
>> Of course the computing world is changing, and IEEE format may well 
>> be  ubiquitous by now. Vaxes must be in the museum by now. But how 
>> about  mainframes? IBM mainframes didn't use IEEE when I used them 
>> (last time  15 years ago), and they are still around, possibly 
>> compatible to their  ancestors.
>
> I found the following piece, written about 6 years ago interesting:
>
> http://www.research.ibm.com/journal/rd/435/schwarz.html
>
> Basically, it states that chips in newer IBM mainframes support the 
> IEEE 754 standard.
>
>>
>> Another detail to consider is that although most machines use the 
>> IEEE  representation, hardly any respects the IEEE rules for floating 
>> point  operations in all detail. In particular, trusting that Inf and 
>> NaN will  be treated as IEEE postulates is a risky business.
>
> But, this can be handled with platform-dependendent C-code when and if 
> problems arise.
> -Travis
>
>
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real 
> users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion





More information about the Numpy-discussion mailing list