[SciPy-Dev] Lapack alignment test in linalg failing

Anne Archibald peridot.faceted@gmail....
Mon Apr 12 14:19:56 CDT 2010


On 12 April 2010 12:22, Warren Weckesser <warren.weckesser@enthought.com> wrote:
> I created a ticket: http://projects.scipy.org/scipy/ticket/1152
>
> If any of the original authors or editors of the code that tests
> misaligned arrays are still around, some feedback would be appreciated,
> especially about why the misalignment is being done twice.  I'm not sure
> if that is intentional.

That code is my fault, and it's a mess. I think the correct way to
handle this is to catch the "NaNs/Infs present" exception, or possibly
to edit the misaligned arrays to remove any NaNs/Infs. Unfortunately
the bug this was attempting to tickle is deep in the bowels of ATLAS,
and is very difficult to trigger correctly; in particular, an identity
matrix does not trigger the bug. This is the reason S is misaligned
twice: it takes that kind of bizarre data to trigger the bug. Since
the symptom of the bug is a segfault, raising a "no NaNs" error
qualifies as a pass (though I'm a little surprised you got any NaNs -
what platform are you using?)

Anne

> Cheers,
>
> Warren
>
>
> Warren Weckesser wrote:
>> I am getting a failure of an alignment test in test_decomp.py in the linalg
>> test suite.  I just made several changes in linalg, but this failure has
>> been occurring since before those changes.
>>
>> Here is the error message:
>>
>> ======================================================================
>> ERROR: test_decomp.test_lapack_misaligned(<function solve at 0x4160c70>,
>> (array([[  1.73394741e-255,   8.18880997e-217,   4.02522535e-178,
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>>   File
>> "/Library/Frameworks/Python.framework/Versions/6.1/lib/python2.6/site-packages/nose/case.py",
>> line 183, in runTest
>>     self.test(*self.arg)
>>   File
>> "/Users/warren/scipy_src_cho_solveh_banded/scipy/linalg/tests/test_decomp.py",
>> line 1074, in check_lapack_misaligned
>>     func(*a,**kwargs)
>>   File
>> "/Users/warren/scipy_src_cho_solveh_banded/build/lib.macosx-10.5-i386-2.6/scipy/linalg/basic.py",
>> line 47, in solve
>>     a1, b1 = map(asarray_chkfinite,(a,b))
>>   File
>> "/Library/Frameworks/Python.framework/Versions/6.1/lib/python2.6/site-packages/numpy/lib/function_base.py",
>> line 586, in asarray_chkfinite
>>     raise ValueError, "array must not contain infs or NaNs"
>> ValueError: array must not contain infs or NaNs
>>
>> ----------------------------------------------------------------------
>>
>> The file test_decomp.py in linalg/tests contains the following code:
>>
>> -----
>> def check_lapack_misaligned(func, args, kwargs):
>>     args = list(args)
>>     for i in range(len(args)):
>>         a = args[:]
>>         if isinstance(a[i],np.ndarray):
>>             # Try misaligning a[i]
>>             aa = np.zeros(a[i].size*a[i].dtype.itemsize+8, dtype=np.uint8)
>>             aa = np.frombuffer(aa.data, offset=4, count=a[i].size,
>> dtype=a[i].dtype)
>>             aa.shape = a[i].shape
>>             aa[...] = a[i]
>>             a[i] = aa
>>             func(*a,**kwargs)
>>             if len(a[i].shape)>1:
>>                 a[i] = a[i].T
>>                 func(*a,**kwargs)
>>
>>
>> def test_lapack_misaligned():
>>     M = np.eye(10,dtype=float)
>>     R = np.arange(100)
>>     R.shape = 10,10
>>     S = np.arange(20000,dtype=np.uint8)
>>     S = np.frombuffer(S.data, offset=4, count=100, dtype=np.float)
>>     S.shape = 10, 10
>>     b = np.ones(10)
>>     v = np.ones(3,dtype=float)
>>     LU, piv = lu_factor(S)
>>     for (func, args, kwargs) in [
>>             (eig,(S,),dict(overwrite_a=True)), # crash
>>             (eigvals,(S,),dict(overwrite_a=True)), # no crash
>>             (lu,(S,),dict(overwrite_a=True)), # no crash
>>             (lu_factor,(S,),dict(overwrite_a=True)), # no crash
>>             (lu_solve,((LU,piv),b),dict(overwrite_b=True)),
>>             (solve,(S,b),dict(overwrite_a=True,overwrite_b=True)),
>>             (svd,(M,),dict(overwrite_a=True)), # no crash
>>             (svd,(R,),dict(overwrite_a=True)), # no crash
>>             (svd,(S,),dict(overwrite_a=True)), # crash
>>             (svdvals,(S,),dict()), # no crash
>>             (svdvals,(S,),dict(overwrite_a=True)), #crash
>>             (cholesky,(M,),dict(overwrite_a=True)), # no crash
>>             (qr,(S,),dict(overwrite_a=True)), # crash
>>             (rq,(S,),dict(overwrite_a=True)), # crash
>>             (hessenberg,(S,),dict(overwrite_a=True)), # crash
>>             (schur,(S,),dict(overwrite_a=True)), # crash
>>             ]:
>>         yield check_lapack_misaligned, func, args, kwargs
>> ---
>>
>> The error occurs when `solve` is called from within
>> check_lapack_misaligned.  Since `solve` has two array arguments,
>> check_lapack_misaligned will try to call `solve` four times.
>> The problem is that the first call results in NaNs in the arrays,
>> which causes the next call to to `solve` to fail.  Apparently the NaNs
>> result from the data in the array being random junk with exponents
>> ranging all over.
>>
>> If I add these lines after setting `S.shape` in test_lapack_misaligned,
>> the error does not occur:
>>
>>     S[...] = 0.0
>>     S[range(10),range(10)] = 1.0
>>
>> In other words, keep S misaligned, but make it the identity matrix.
>>
>> Is that a "saner" test?
>>
>> The error can also be avoided by setting the overwrite arguments
>> to False.  However, I don't know if the values of these arguments
>> are an important part of the test.
>>
>> I don't really understand the logic in these two functions.
>> test_lapack_misaligned misaligns `S`, but then check_lapack_misaligned
>> further misaligns its arguments.  Why do this twice?
>>
>> Warren
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev@scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


More information about the SciPy-Dev mailing list