[Numpy-discussion] Bizarre errors with byteswapping, complex256, PPC

Charles R Harris charlesr.harris@gmail....
Thu Jun 21 01:57:57 CDT 2012


On Thu, Jun 21, 2012 at 12:11 AM, Matthew Brett <matthew.brett@gmail.com>wrote:

> Hi,
>
> On Wed, Jun 20, 2012 at 10:43 PM, Charles R Harris
> <charlesr.harris@gmail.com> wrote:
> >
> >
> > On Wed, Jun 20, 2012 at 4:11 PM, Matthew Brett <matthew.brett@gmail.com>
> > wrote:
> >>
> >> Hi,
> >>
> >> On Wed, Jun 20, 2012 at 3:05 PM, Charles R Harris
> >> <charlesr.harris@gmail.com> wrote:
> >> >
> >> >
> >> > On Wed, Jun 20, 2012 at 4:00 PM, Matthew Brett <
> matthew.brett@gmail.com>
> >> > wrote:
> >> >>
> >> >> Hi,
> >> >>
> >> >> On Wed, Jun 20, 2012 at 1:56 PM, Travis Oliphant <
> travis@continuum.io>
> >> >> wrote:
> >> >> > This looks like a problem with comparisons of floating point
> numbers
> >> >> > rather than a byteswapping problem per-say.   Try to use an almost
> >> >> > equal
> >> >> > comparison instead.
> >> >>
> >> >> Is that right - that the byteswapped versions might not be strictly
> >> >> equal to identical numbers but not byteswapped?
> >> >>
> >> >> But I should maybe have been clearer - they also subtract wrongly:
> >> >>
> >> >> <script>
> >> >> import numpy as np
> >> >>
> >> >> arr = np.arange(10, dtype=np.complex256)
> >> >> bs_arr = arr.byteswap().newbyteorder('S')
> >> >> print arr
> >> >> print bs_arr
> >> >> print arr - bs_arr
> >> >> print arr - bs_arr
> >> >> print arr - bs_arr
> >> >> </script>
> >> >>
> >> >> (np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py
> >> >> [ 0.0+0.0j  1.0+0.0j  2.0+0.0j  3.0+0.0j  4.0+0.0j  5.0+0.0j
>  6.0+0.0j
> >> >>  7.0+0.0j  8.0+0.0j  9.0+0.0j]
> >> >> [ 0.0+0.0j  1.0+0.0j  2.0+0.0j  3.0+0.0j  4.0+0.0j  5.0+0.0j
>  6.0+0.0j
> >> >>  7.0+0.0j  8.0+0.0j  9.0+0.0j]
> >> >> [ 0.0+0.0j  1.0+0.0j  2.0+0.0j  3.0+0.0j  4.0+0.0j  5.0+0.0j
>  6.0+0.0j
> >> >>  7.0+0.0j  8.0+0.0j  9.0+0.0j]
> >> >> [ 0.0+0.0j  1.0+0.0j  2.0+0.0j  3.0+0.0j  4.0+0.0j  5.0+0.0j
>  6.0+0.0j
> >> >>  7.0+0.0j  8.0+0.0j  9.0+0.0j]
> >> >> [ 0.0+0.0j  1.0+0.0j  2.0+0.0j  3.0+0.0j  4.0+0.0j  5.0+0.0j
>  6.0+0.0j
> >> >>  7.0+0.0j  8.0+0.0j  9.0+0.0j]
> >> >>
> >> >> (wrong)
> >> >>
> >> >> (np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py
> >> >> [ 0.0+0.0j  1.0+0.0j  2.0+0.0j  3.0+0.0j  4.0+0.0j  5.0+0.0j
>  6.0+0.0j
> >> >>  7.0+0.0j  8.0+0.0j  9.0+0.0j]
> >> >> [ 0.0+0.0j  1.0+0.0j  2.0+0.0j  3.0+0.0j  4.0+0.0j  5.0+0.0j
>  6.0+0.0j
> >> >>  7.0+0.0j  8.0+0.0j  9.0+0.0j]
> >> >> [ 0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j
>  0.0+0.0j
> >> >>  0.0+0.0j  0.0+0.0j  0.0+0.0j]
> >> >> [ 0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j
>  0.0+0.0j
> >> >>  0.0+0.0j  0.0+0.0j  0.0+0.0j]
> >> >> [ 0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j
>  0.0+0.0j
> >> >>  0.0+0.0j  0.0+0.0j  0.0+0.0j]
> >> >>
> >> >> (right)
> >> >>
> >> >> See you,
> >> >>
> >> >
> >> > Long doubles on PPC consist of two doubles, so I expect you need to
> swap
> >> > both doubles instead of 16 bytes. Strictly speaking, numpy doesn't
> >> > support
> >> > non ieee floats.
> >>
> >> Well - the byteswapping appears to be correct in that the array is
> >> displayed with the correct values, but then, when doing a subtraction
> >> on the array, most of the time it is incorrect, but whether it is
> >> correct or incorrect, appears to be random even with the same
> >> variables and memory.
> >>
> >> Float128 and other numpy dtypes appear to be correct using the same
> tests.
> >
> >
> > Thinking about it, that makes sense because the swapped version is
> probably
> > incorrect ;) That is, the PPC was (is?) selectable to run either little
> > endian or big endian, so the real test would be if long doubles were
> > portable between machines set up different ways. The only machines I
> know of
> > are little endian, but IIRC, there was at least one brand that was big
> > endian. However, I suspect we are just reversing the whole 16 bytes, so
> even
> > though that is pretty much a meaningless thing to do, it should work...
> >
> > Is this something that only happens on 32 bit machines?
>
> The PPC machines I have record themselves as big endian (2 running OSX
> and 1 running Debian wheezy).
>
> I can only get the Debian wheezy machine to misbehave in this way.
>
> The original report of the problem was on a POWER7 machine running
> Debian - wheezy I think.  I guess this is 64 bit - Yarik - do you
> know?
>
>
Looks like I got little/big endian reversed. Anyway, this is very strange.
Same compilers on both machines? I don't understand why this wouldn't show
up on other machines with float128, and in particular why is should only
happen for complex256 and Debian. What is the extended precision type in
OSX on PPC?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20120621/402949cb/attachment.html 


More information about the NumPy-Discussion mailing list