[NumPy-Tickets] [NumPy] #1631: Solaris10/SPARC/gcc-4.5.1/32bit: SIGBUS in test_mrecords.py

[NumPy Trac]

#1631: Solaris10/SPARC/gcc-4.5.1/32bit: SIGBUS in test_mrecords.py
--------------------+-------------------------------------------------------
 Reporter:  nics    |       Owner:  somebody
     Type:  defect  |      Status:  new     
 Priority:  normal  |   Milestone:  1.5.1   
Component:  Other   |     Version:  1.5.0   
 Keywords:          |  
--------------------+-------------------------------------------------------
 Hi everybody,

 for numpy-1.5.0, the testsuite crashes here with an SIGBUS here:
 "Test filled w/ flexible dtype ... Bus Error (core dumped)"
 (numpy-1.5.0/numpy/ma/tests/test_core.py)
 and here:
 "Test that 'exotic' formats are processed properly ... Bus Error (core
 dumped)" (numpy-1.5.0/numpy/ma/tests/test_mrecords.py).

 The problem is that SPARC processors require alignment, that is 8 byte
 values have to be aligned at 8 byte memory boundraries.
 The good point is: numpy knows that the array is misaligned (verified
 using gdb) and thus, the fix is easy (see attached diff
 02_put_mask_only_on_behaved_arrays.diff): The check for ISCONTINOUS isn't
 enough in PyArray_PutMask (item_selection.c), also check for ISBEHAVED (=
 ISALIGNED && ISWRITABLE).

 #############

 But now, another problem arises:
 a SIGSEGV here:
 "Tests fields retrieval"
 (numpy-1.5.0/numpy/ma/tests/test_mrecords.py:77).

 The problem is in
 _copy_same_shape(numpy-1.5.0/numpy/core/src/multiarrayctors.c:732):
 dest->dimensions == NULL.
 Please also note that maxaxis == -1 at that point.
 Check the attached diff "04_copy_from_same_shape_zerodim_fix.diff" for
 details. There's one point in the diff I'm unsure about: The right
 position of
  PyArray_INCREF(src);
  PyArray_XDECREF(dest);
 I tried to resemble the original "logical" position, but since I have no
 clue about Python's reference counting, please have a look.

 #############

 While debugging the last SIGSEGV-issue, I stumbled over another mistake
 (at least I think so):

 In PyArray_IterAllButAxis
 (numpy-1.5.0/numpy/core/src/multiarray/iterators.c), the minaxis won't be
 set if the first nonzero stride is the smallest one.
 See attached diff (03_fix_iterallbutaxis_minstride_search.diff) for a fix.

 #############

 The reason why I poste those three issues into one report is that the
 testsuite still doesn't succeed and thus, I'm unsure if I've broken sth.
 with my diffs.

 What I get now is:
 ======================================================================
 FAIL: Ticket #1299 second test
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File
 "/pf/m/m222086/xas/solaris10/python2/python-2.7-ve0-gcc/lib/python2.7/sit
 e-packages/numpy/core/tests/test_regression.py", line 1255, in
 test_structured_a
 rrays_with_objects2
     assert sys.getrefcount(strb) == numb
 AssertionError:
     7 = <module 'sys' (built-in)>.getrefcount('aaaa')
     array([[(0L, 'aaaa'), (0L, 'aaaa')]],
           dtype=[('f0', '>i8'), ('f1', '|O4')]) = <module 'numpy' from
 '/pf/m/m2
 22086/xas/solaris10/python2/python-2.7-ve0-gcc/lib/python2.7/site-
 packages/numpy
 /__init__.pyc'>.array([[(0,'aaaa'),(1,'bbbb')]], 'i8,O')
 >>  array([[(0L, 'aaaa'), (0L, 'aaaa')]],
           dtype=[('f0', '>i8'), ('f1', '|O4')])[array([[(0L, 'aaaa'), (0L,
 'aaaa
 ')]],
           dtype=[('f0', '>i8'), ('f1', '|O4')]).nonzero()] = array([[(0L,
 'aaaa'
 ), (0L, 'aaaa')]],
           dtype=[('f0', '>i8'), ('f1', '|O4')]).ravel()[:1]
     assert <module 'sys' (built-in)>.getrefcount('bbbb') == 7
     assert <module 'sys' (built-in)>.getrefcount('aaaa') == 7 + 2


 ======================================================================
 FAIL: Test filled w/ mvoid
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File
 "/pf/m/m222086/xas/solaris10/python2/python-2.7-ve0-gcc/lib/python2.7/sit
 e-packages/numpy/ma/tests/test_core.py", line 518, in test_filled_w_mvoid
     assert_equal(tuple(test), (1, default_fill_value(1.)))
   File
 "/pf/m/m222086/xas/solaris10/python2/python-2.7-ve0-gcc/lib/python2.7/sit
 e-packages/numpy/ma/testutils.py", line 94, in assert_equal
     return _assert_equal_on_sequences(actual, desired, err_msg='')
   File
 "/pf/m/m222086/xas/solaris10/python2/python-2.7-ve0-gcc/lib/python2.7/sit
 e-packages/numpy/ma/testutils.py", line 66, in _assert_equal_on_sequences
     assert_equal(actual[k], desired[k], 'item=%r\n%s' % (k,err_msg))
   File
 "/pf/m/m222086/xas/solaris10/python2/python-2.7-ve0-gcc/lib/python2.7/sit
 e-packages/numpy/ma/testutils.py", line 98, in assert_equal
     raise AssertionError(msg)
 AssertionError:
 Items are not equal:
 item=1

  ACTUAL: 2.0
  DESIRED: 1e+20
 >>  raise AssertionError('\nItems are not equal:\nitem=1\n\n ACTUAL: 2.0\n
 DESIR
 ED: 1e+20')


 At least not segfaults/bus errors anymore ;).
 Btw.: I don't know what mvoid is, but have a look at the following:

 myuid@myhost:~$ ~/xas/solaris10/python2/python-2.7-ve0-gcc/bin/python
 Python 2.7 (r27:82500, Oct  9 2010, 17:26:38)
 [GCC 4.5.1] on sunos5
 Type "help", "copyright", "credits" or "license" for more information.
 >>> import numpy as np
 >>> import numpy.ma as ma
 >>> import numpy.ma.core
 >>> from numpy.ma.core import *
 >>> x = ma.array([(1,2.)], mask=[(0,1)], dtype=[('a', int), ('b', float)])
 >>> print x.filled()
 [(1, 1e+20)]
 >>> x = mvoid((1,2.), mask=[(0,1)], dtype=[('a', int), ('b', float)])
 >>> print x.filled()
 (1, 2.0)
 >>>

 Since the non-working mvoid-mask could have an impact on results, I cannot
 release that build to our the scientists at our site. Do you have any idea
 what the issue could be?
 Do you know what a mvoid is? I can't find any documentation about it. I'm
 seriously thinking about just removing that class from my numpy build (If
 I knew that there weren't any dependants outside of numpy)...

 Wishes

 Nicolai

-- 
Ticket URL: <http://projects.scipy.org/numpy/ticket/1631>
NumPy <http://projects.scipy.org/numpy>
My example project