[Numpy-discussion] Questions about the array interface.

Todd Miller jmiller at stsci.edu
Sat Apr 9 16:18:00 CDT 2005


On Sat, 2005-04-09 at 12:35 -0700, Andrew Straw wrote:
> Here's an email Todd Miller sent me (I hoped he'd send it directly to 
> the list, but I'll forward it.  Todd, I hope you don't mind.)

No, I don't mind.  I intended to send it to the list but left in a rush
this morning.  

Todd

> 
> > On Fri, 2005-04-08 at 15:46 -0700, Andrew Straw wrote:
> >> Hi Todd,
> >>
> >> Could you join in on this thread?  I think you wrote the ieeespecial
> >> stuff in numarray, so it's clear you have a much better understanding 
> >> of
> >> the issues than I do...
> >>
> >> Cheers!
> >> Andrew
> >
> > My own understanding is limited,  but I can say a few things that might
> > make the status of numarray clearer.  My assumptions for numarray were
> > that:
> >
> > 1. Floating point values are 32-bit or 64-bit entities which are stored
> > in IEEE-754 format.  This is a basic assumption of numarray.ieeespecial
> > so I expect it simply won't work on a VAX.  There's no checking for
> > this.
> >
> > 2. The platforms that I care about,  AMD/Intel Windows/Linux, PowerPC
> > OS-X, and Ultra-SPARC Solaris,  all seem to provide IEEE-754 floating
> > point.  ieeespecial has been tested to work there.
> >
> > 3. I viewed IEEE-754 floating point numbers as 32-bit or 64-bit 
> > unsigned
> > integers,  and contiguous ranges on those integers are used to 
> > represent
> > special values like NAN and INF.  Platform byte ordering for the
> > IEEE-754 floating point numbers mirrors byte ordering for integers so
> > the ieeespecial NAN detection code works in a cross platform way *and*
> > values exported from one IEEE-754 platform will work with ieeespecial
> > when imported on another.  It's important to note that special values
> > are not unique:  there is no single NAN value;  it's a bit range.
> >
> > 4. numarray leaks IEEE-754 special values out into Python floating 
> > point
> > scalars.  This may be bad form.  I do this because (1) they repr
> > understandably if not in a platform independent way and (2) people need
> > to get at them.  I noticed recently that ieeespecial.nan ==
> > ieeespecial.nan returns incorrect answers (True!) for Python-2.3 and
> > correct ones (False) for Python-2.4.  I haven't looked at what the 
> > array
> > version does yet:  array(nan) == array(nan).  The point to be taken 
> > from
> > this is that the level at which numarray ieee special value handling
> > works or doesn't work is really restricted to (1) detecting certain
> > ieee-754 bit ranges (2) the basic behavior of C code for C89 complilers
> > for array code (no guarantees) (3) the behavior of Python itself
> > (improving).
> >
> > In the context of the array protocol (looking very nice by the way) my
> > thinking is that non-IEEE-754 floating point could be described with 
> > bit
> > fields and that the current type codes should mean IEEE-754.
> >
> > Some minor things I noticed in the array interface:
> >
> > 1. The packing order of bit fields is not clear.  In C,  my experience
> > is that some compilers pack bit structs towards the higher order bits 
> > of
> > an integer,  and some towards the lower.  More info to clarify that
> > would be helpful.
> >
> > 2.  I saw no mention that we're talking about a protocol.  I'm sure
> > that's clear to everyone following this discussion closely,  but I
> > didn't see it in the spec.  It might make sense to allude to the C
> > helper functions and potential for additions to the Python type struct
> > even if they're not spelled out.
> >
> > Regards,
> > Todd
> 
> 
> On Apr 9, 2005, at 9:54 AM, Travis Oliphant wrote:
> 
> > konrad.hinsen at laposte.net wrote:
> >
> >> On 09.04.2005, at 01:04, Scott Gilbert wrote:
> >>
> >>> I think something we've been assuming is that the array data is  
> >>> basically
> >>> IEEE-754 compliant (maybe it needs to be byteswapped).  If that's 
> >>> not  true,
> >>> then we're going to need some new typecodes.  We're not supporting 
> >>> the
> >>> ability to pass VAX floating point around (Are we????).
> >>
> >
> > No, in moving from the struct modules character codes we are trying to 
> > do something more platform independent because it is very likely that 
> > different platforms will want to exchange binary data.   IEEE-754 is a 
> > great standard to build an interface around.   Data sharing was the 
> > whole reason the standard emerged and a lot of companies got on board.
> >
> >>
> >> This discussion has been coming up regularly for a few years. Until 
> >> now  the concensus has always been that Python should make no 
> >> assumptions  that go beyond what a C compiler can promise. Which 
> >> means no  assumptions about floating-point representation.
> >>
> >> Of course the computing world is changing, and IEEE format may well 
> >> be  ubiquitous by now. Vaxes must be in the museum by now. But how 
> >> about  mainframes? IBM mainframes didn't use IEEE when I used them 
> >> (last time  15 years ago), and they are still around, possibly 
> >> compatible to their  ancestors.
> >
> > I found the following piece, written about 6 years ago interesting:
> >
> > http://www.research.ibm.com/journal/rd/435/schwarz.html
> >
> > Basically, it states that chips in newer IBM mainframes support the 
> > IEEE 754 standard.
> >
> >>
> >> Another detail to consider is that although most machines use the 
> >> IEEE  representation, hardly any respects the IEEE rules for floating 
> >> point  operations in all detail. In particular, trusting that Inf and 
> >> NaN will  be treated as IEEE postulates is a risky business.
> >
> > But, this can be handled with platform-dependendent C-code when and if 
> > problems arise.
> > -Travis
> >
> >
> >
> >
> > -------------------------------------------------------
> > SF email is sponsored by - The IT Product Guide
> > Read honest & candid reviews on hundreds of IT Products from real 
> > users.
> > Discover which products truly live up to the hype. Start reading now.
> > http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> > _______________________________________________
> > Numpy-discussion mailing list
> > Numpy-discussion at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 
> 
> 
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion





More information about the Numpy-discussion mailing list