[Numpy-discussion] Questions about the array interface.
jmiller at stsci.edu
Sat Apr 9 16:18:00 CDT 2005
On Sat, 2005-04-09 at 12:35 -0700, Andrew Straw wrote:
> Here's an email Todd Miller sent me (I hoped he'd send it directly to
> the list, but I'll forward it. Todd, I hope you don't mind.)
No, I don't mind. I intended to send it to the list but left in a rush
> > On Fri, 2005-04-08 at 15:46 -0700, Andrew Straw wrote:
> >> Hi Todd,
> >> Could you join in on this thread? I think you wrote the ieeespecial
> >> stuff in numarray, so it's clear you have a much better understanding
> >> of
> >> the issues than I do...
> >> Cheers!
> >> Andrew
> > My own understanding is limited, but I can say a few things that might
> > make the status of numarray clearer. My assumptions for numarray were
> > that:
> > 1. Floating point values are 32-bit or 64-bit entities which are stored
> > in IEEE-754 format. This is a basic assumption of numarray.ieeespecial
> > so I expect it simply won't work on a VAX. There's no checking for
> > this.
> > 2. The platforms that I care about, AMD/Intel Windows/Linux, PowerPC
> > OS-X, and Ultra-SPARC Solaris, all seem to provide IEEE-754 floating
> > point. ieeespecial has been tested to work there.
> > 3. I viewed IEEE-754 floating point numbers as 32-bit or 64-bit
> > unsigned
> > integers, and contiguous ranges on those integers are used to
> > represent
> > special values like NAN and INF. Platform byte ordering for the
> > IEEE-754 floating point numbers mirrors byte ordering for integers so
> > the ieeespecial NAN detection code works in a cross platform way *and*
> > values exported from one IEEE-754 platform will work with ieeespecial
> > when imported on another. It's important to note that special values
> > are not unique: there is no single NAN value; it's a bit range.
> > 4. numarray leaks IEEE-754 special values out into Python floating
> > point
> > scalars. This may be bad form. I do this because (1) they repr
> > understandably if not in a platform independent way and (2) people need
> > to get at them. I noticed recently that ieeespecial.nan ==
> > ieeespecial.nan returns incorrect answers (True!) for Python-2.3 and
> > correct ones (False) for Python-2.4. I haven't looked at what the
> > array
> > version does yet: array(nan) == array(nan). The point to be taken
> > from
> > this is that the level at which numarray ieee special value handling
> > works or doesn't work is really restricted to (1) detecting certain
> > ieee-754 bit ranges (2) the basic behavior of C code for C89 complilers
> > for array code (no guarantees) (3) the behavior of Python itself
> > (improving).
> > In the context of the array protocol (looking very nice by the way) my
> > thinking is that non-IEEE-754 floating point could be described with
> > bit
> > fields and that the current type codes should mean IEEE-754.
> > Some minor things I noticed in the array interface:
> > 1. The packing order of bit fields is not clear. In C, my experience
> > is that some compilers pack bit structs towards the higher order bits
> > of
> > an integer, and some towards the lower. More info to clarify that
> > would be helpful.
> > 2. I saw no mention that we're talking about a protocol. I'm sure
> > that's clear to everyone following this discussion closely, but I
> > didn't see it in the spec. It might make sense to allude to the C
> > helper functions and potential for additions to the Python type struct
> > even if they're not spelled out.
> > Regards,
> > Todd
> On Apr 9, 2005, at 9:54 AM, Travis Oliphant wrote:
> > konrad.hinsen at laposte.net wrote:
> >> On 09.04.2005, at 01:04, Scott Gilbert wrote:
> >>> I think something we've been assuming is that the array data is
> >>> basically
> >>> IEEE-754 compliant (maybe it needs to be byteswapped). If that's
> >>> not true,
> >>> then we're going to need some new typecodes. We're not supporting
> >>> the
> >>> ability to pass VAX floating point around (Are we????).
> > No, in moving from the struct modules character codes we are trying to
> > do something more platform independent because it is very likely that
> > different platforms will want to exchange binary data. IEEE-754 is a
> > great standard to build an interface around. Data sharing was the
> > whole reason the standard emerged and a lot of companies got on board.
> >> This discussion has been coming up regularly for a few years. Until
> >> now the concensus has always been that Python should make no
> >> assumptions that go beyond what a C compiler can promise. Which
> >> means no assumptions about floating-point representation.
> >> Of course the computing world is changing, and IEEE format may well
> >> be ubiquitous by now. Vaxes must be in the museum by now. But how
> >> about mainframes? IBM mainframes didn't use IEEE when I used them
> >> (last time 15 years ago), and they are still around, possibly
> >> compatible to their ancestors.
> > I found the following piece, written about 6 years ago interesting:
> > http://www.research.ibm.com/journal/rd/435/schwarz.html
> > Basically, it states that chips in newer IBM mainframes support the
> > IEEE 754 standard.
> >> Another detail to consider is that although most machines use the
> >> IEEE representation, hardly any respects the IEEE rules for floating
> >> point operations in all detail. In particular, trusting that Inf and
> >> NaN will be treated as IEEE postulates is a risky business.
> > But, this can be handled with platform-dependendent C-code when and if
> > problems arise.
> > -Travis
> > -------------------------------------------------------
> > SF email is sponsored by - The IT Product Guide
> > Read honest & candid reviews on hundreds of IT Products from real
> > users.
> > Discover which products truly live up to the hype. Start reading now.
> > http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> > _______________________________________________
> > Numpy-discussion mailing list
> > Numpy-discussion at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
More information about the Numpy-discussion