[Numpy-discussion] FWIW: "regressions" of dependees of numpy 1.7.0b1

Yaroslav Halchenko lists@onerussian....
Thu Sep 6 11:24:40 CDT 2012


preamble:  the bug here seems to be due to incorrect
           np.asarray(ctypes.cdouble array)

ok -- I tried with a debug build of python and -O0 build of numpy, and
the same old valgrind... this time  valgrind is silent BUT then python itself
says 

test_simple (mvpa2.tests.test_procrust.ProcrusteanMapperTests) ... XXX undetected error
ERROR

and nose failure  (now I see it) has nothing actually to do with the operation
on the object but just get reported to the first line after the function call
where I guess a problematic object was created... 


======================================================================
ERROR: test_simple (mvpa2.tests.test_procrust.ProcrusteanMapperTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/yoh/proj/pymvpa/pymvpa/mvpa2/testing/sweepargs.py", line 67, in do_sweep
    method(*args_, **kwargs_)
  File "/home/yoh/proj/pymvpa/pymvpa/mvpa2/testing/sweepargs.py", line 67, in do_sweep
    method(*args_, **kwargs_)
  File "/home/yoh/proj/pymvpa/pymvpa/mvpa2/testing/tools.py", line 179, in newfunc
    return func(*arg, **kwargs)
  File "/home/yoh/proj/pymvpa/pymvpa/mvpa2/tests/test_procrust.py", line 67, in test_simple
    pm.train(ds)
  File "/home/yoh/proj/pymvpa/pymvpa/mvpa2/base/learner.py", line 119, in train
    result = self._train(ds)
  File "/home/yoh/proj/pymvpa/pymvpa/mvpa2/mappers/procrustean.py", line 161, in _train
    print "------------"
TypeError: object of type 'float' has no len()


that function is fancy only in that it uses ctypes to call a function
from the cdll.LoadLibrary('liblapack.so'):
https://github.com/PyMVPA/PyMVPA/blob/HEAD/mvpa2/support/lapack_svd.py but even
if I comment out that call to lapacklib.  it still screws up the same way -- so
it has to do with those variable definitions before then I guess... 

ok -- it boils down to numpy.asarray(s) in return statement... I improved it
with a printout now where I assigned constructed array to a variable first

NB if I swap print lines, it would lead me to crash above, but with this order
   -- it manages to continue without crashing BUT showing incorrect values.

        s_arr = numpy.asarray(s)
        print "s_arr", s_arr
        print "s:", s
        return vt, s_arr, u

so it gives me

    $> MVPA_SEED=1928295852 `which nosetests` -s -v mvpa2/tests/test_procrust.py
    T: MVPA_SEED=1928295852
    test_simple (mvpa2.tests.test_procrust.ProcrusteanMapperTests) ... VERSION
    1.7.0rc1.dev-ea23de8
    s_arr [[  6.90689888e-310   1.83759219e-316]
     [ -3.16388621e+134  -3.16388621e+134]]
    s: <mvpa2.support.lapack_svd.c_double_Array_2 object at 0x28c4ae0>
    FAIL

    ======================================================================
    FAIL: test_simple (mvpa2.tests.test_procrust.ProcrusteanMapperTests)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/yoh/proj/pymvpa/pymvpa/mvpa2/testing/tools.py", line 179, in newfunc
        return func(*arg, **kwargs)
      File "/home/yoh/proj/pymvpa/pymvpa/mvpa2/tests/test_procrust.py", line 93, in test_simple
        "perfectly. Now got d scale*R=%g" % dsR)
    AssertionError: 
     Single scenario lead to failures of unittest test_simple:
      on
        oblique=False :
         
     Single scenario lead to failures of unittest test_simple:
      on
        svd=dgesvd :
         We should have got reconstructed rotation+scaling perfectly. Now got d scale*R=1.34232e+134



    ----------------------------------------------------------------------
    Ran 1 test in 0.222s

    FAILED (failures=1)
    [161643 refs]

when with 1.6.2 (using the same seed, so if numpy's RNG didn't change -- data should be the same):

    $> MVPA_SEED=1928295852 nosetests -s -v mvpa2/tests/test_procrust.py
    T: MVPA_SEED=1928295852
    test_simple (mvpa2.tests.test_procrust.ProcrusteanMapperTests) ... VERSION
    1.6.2
    s_arr [0.775771814652 0.224228185348]
    s: <mvpa2.support.lapack_svd.c_double_Array_2 object at 0xad65d40>
    s_arr [0.775771814652 0.224228185348]
    s: <mvpa2.support.lapack_svd.c_double_Array_2 object at 0xad65c20>
    s_arr [0.365993114976 0.101324191354 0.0959100894799 0.0861936270658
     0.0712098129231 0.0694159685405 0.0636016740376 0.056058975141
     0.0481853714064 0.042107175076]
    s: <mvpa2.support.lapack_svd.c_double_Array_10 object at 0xad65d40>
    s_arr [0.365993114976 0.101324191354 0.0959100894799 0.0861936270658
     0.0712098129231 0.0694159685405 0.0636016740376 0.056058975141
     0.0481853714064 0.042107175076]
    s: <mvpa2.support.lapack_svd.c_double_Array_10 object at 0xad65c20>
    s_arr [0.775771814652 0.224228185348 0.0]
    s: <mvpa2.support.lapack_svd.c_double_Array_3 object at 0xad65d40>
    s_arr [0.775771814652 0.224228185348 0.0]
    s: <mvpa2.support.lapack_svd.c_double_Array_3 object at 0xad65c20>
    s_arr [0.703907195999 0.200251541932 0.0]
    s: <mvpa2.support.lapack_svd.c_double_Array_3 object at 0xad65d40>
    s_arr [0.703907195999 0.200251541932 0.0]
    s: <mvpa2.support.lapack_svd.c_double_Array_3 object at 0xad65c20>
    ok

    ----------------------------------------------------------------------
    Ran 1 test in 7.031s

    OK
    MVPA_SEED=1928295852 nosetests -s -v mvpa2/tests/test_procrust.py  7,14s user 0,79s system 96% cpu 8,219 total

which immediately shows that np.asarray created a 2d array whenever it
should have been a 1d (original definition of s is s=(c_double*min(x,y))())

ok -- added printing of dtype of that array -- with 1.6.2 it is reported  as
object while with 1.7.0b1 -- float64...

from here I pass it onto experts! ;)

On Thu, 06 Sep 2012, Yaroslav Halchenko wrote:


> On Thu, 06 Sep 2012, Aron Ahmadia wrote:

> >    Are you running the valgrind test with the Python suppression
> >    file:�[1]http://svn.python.org/projects/python/trunk/Misc/valgrind-python.supp

> yes -- on Debian there is /usr/lib/valgrind/python.supp  which  comes
> with python package and I believe enabled by default, and it is
> identical to above (just dynamic library versions different) but it
> still produces lots of false positives -- IIRC it needs additional tune
> ups per architecture etc... I just ignored those messages
> "manually" and listed the relevant one which comes from numpy
> functionality.
-- 
Yaroslav O. Halchenko
Postdoctoral Fellow,   Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        


More information about the NumPy-Discussion mailing list