[Numpy-discussion] Bizarre errors with byteswapping, complex256, PPC

Matthew Brett matthew.brett@gmail....
Thu Jun 21 01:11:37 CDT 2012


Hi,

On Wed, Jun 20, 2012 at 10:43 PM, Charles R Harris
<charlesr.harris@gmail.com> wrote:
>
>
> On Wed, Jun 20, 2012 at 4:11 PM, Matthew Brett <matthew.brett@gmail.com>
> wrote:
>>
>> Hi,
>>
>> On Wed, Jun 20, 2012 at 3:05 PM, Charles R Harris
>> <charlesr.harris@gmail.com> wrote:
>> >
>> >
>> > On Wed, Jun 20, 2012 at 4:00 PM, Matthew Brett <matthew.brett@gmail.com>
>> > wrote:
>> >>
>> >> Hi,
>> >>
>> >> On Wed, Jun 20, 2012 at 1:56 PM, Travis Oliphant <travis@continuum.io>
>> >> wrote:
>> >> > This looks like a problem with comparisons of floating point numbers
>> >> > rather than a byteswapping problem per-say.   Try to use an almost
>> >> > equal
>> >> > comparison instead.
>> >>
>> >> Is that right - that the byteswapped versions might not be strictly
>> >> equal to identical numbers but not byteswapped?
>> >>
>> >> But I should maybe have been clearer - they also subtract wrongly:
>> >>
>> >> <script>
>> >> import numpy as np
>> >>
>> >> arr = np.arange(10, dtype=np.complex256)
>> >> bs_arr = arr.byteswap().newbyteorder('S')
>> >> print arr
>> >> print bs_arr
>> >> print arr - bs_arr
>> >> print arr - bs_arr
>> >> print arr - bs_arr
>> >> </script>
>> >>
>> >> (np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py
>> >> [ 0.0+0.0j  1.0+0.0j  2.0+0.0j  3.0+0.0j  4.0+0.0j  5.0+0.0j  6.0+0.0j
>> >>  7.0+0.0j  8.0+0.0j  9.0+0.0j]
>> >> [ 0.0+0.0j  1.0+0.0j  2.0+0.0j  3.0+0.0j  4.0+0.0j  5.0+0.0j  6.0+0.0j
>> >>  7.0+0.0j  8.0+0.0j  9.0+0.0j]
>> >> [ 0.0+0.0j  1.0+0.0j  2.0+0.0j  3.0+0.0j  4.0+0.0j  5.0+0.0j  6.0+0.0j
>> >>  7.0+0.0j  8.0+0.0j  9.0+0.0j]
>> >> [ 0.0+0.0j  1.0+0.0j  2.0+0.0j  3.0+0.0j  4.0+0.0j  5.0+0.0j  6.0+0.0j
>> >>  7.0+0.0j  8.0+0.0j  9.0+0.0j]
>> >> [ 0.0+0.0j  1.0+0.0j  2.0+0.0j  3.0+0.0j  4.0+0.0j  5.0+0.0j  6.0+0.0j
>> >>  7.0+0.0j  8.0+0.0j  9.0+0.0j]
>> >>
>> >> (wrong)
>> >>
>> >> (np-devel)[mb312@joshlegacy ~/tmp]$ python funny_bs.py
>> >> [ 0.0+0.0j  1.0+0.0j  2.0+0.0j  3.0+0.0j  4.0+0.0j  5.0+0.0j  6.0+0.0j
>> >>  7.0+0.0j  8.0+0.0j  9.0+0.0j]
>> >> [ 0.0+0.0j  1.0+0.0j  2.0+0.0j  3.0+0.0j  4.0+0.0j  5.0+0.0j  6.0+0.0j
>> >>  7.0+0.0j  8.0+0.0j  9.0+0.0j]
>> >> [ 0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j
>> >>  0.0+0.0j  0.0+0.0j  0.0+0.0j]
>> >> [ 0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j
>> >>  0.0+0.0j  0.0+0.0j  0.0+0.0j]
>> >> [ 0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j  0.0+0.0j
>> >>  0.0+0.0j  0.0+0.0j  0.0+0.0j]
>> >>
>> >> (right)
>> >>
>> >> See you,
>> >>
>> >
>> > Long doubles on PPC consist of two doubles, so I expect you need to swap
>> > both doubles instead of 16 bytes. Strictly speaking, numpy doesn't
>> > support
>> > non ieee floats.
>>
>> Well - the byteswapping appears to be correct in that the array is
>> displayed with the correct values, but then, when doing a subtraction
>> on the array, most of the time it is incorrect, but whether it is
>> correct or incorrect, appears to be random even with the same
>> variables and memory.
>>
>> Float128 and other numpy dtypes appear to be correct using the same tests.
>
>
> Thinking about it, that makes sense because the swapped version is probably
> incorrect ;) That is, the PPC was (is?) selectable to run either little
> endian or big endian, so the real test would be if long doubles were
> portable between machines set up different ways. The only machines I know of
> are little endian, but IIRC, there was at least one brand that was big
> endian. However, I suspect we are just reversing the whole 16 bytes, so even
> though that is pretty much a meaningless thing to do, it should work...
>
> Is this something that only happens on 32 bit machines?

The PPC machines I have record themselves as big endian (2 running OSX
and 1 running Debian wheezy).

I can only get the Debian wheezy machine to misbehave in this way.

The original report of the problem was on a POWER7 machine running
Debian - wheezy I think.  I guess this is 64 bit - Yarik - do you
know?

See you,

Matthew


More information about the NumPy-Discussion mailing list