[Numpy-discussion] Mysterious test_pareto failure on Travis

josef.pktd@gmai... josef.pktd@gmai...
Tue Sep 4 15:48:23 CDT 2012


On Tue, Sep 4, 2012 at 4:37 PM, Nathaniel Smith <njs@pobox.com> wrote:
> On Tue, Sep 4, 2012 at 9:17 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
>> On Tue, Sep 4, 2012 at 12:41 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
>>> On Tue, Sep 4, 2012 at 12:31 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
>>>> On Tue, Sep 4, 2012 at 3:15 AM, Nathaniel Smith <njs@pobox.com> wrote:
>>>>> The last two Travis builds of master have failed consistently with the
>>>>> same error:
>>>>>   http://travis-ci.org/#!/numpy/numpy/builds
>>>>> It looks like a real failure -- we're getting the same error on every
>>>>> build variant, some sort of problem in test_pareto. Example:
>>>>>   http://travis-ci.org/#!/numpy/numpy/jobs/2328823
>>>>>
>>>>> The obvious culprit would be the previous commit, which regenerated
>>>>> mtrand.c with Cython 0.17:
>>>>>   http://github.com/numpy/numpy/commit/cd9092aa71d23359b33e89d938c55fb14b9bf606
>>>>>
>>>>> What's weird, though, is that that commit passed just fine on Travis:
>>>>>   http://travis-ci.org/#!/numpy/numpy/builds/2313124
>>>>>
>>>>> It's just the two commits since then that failed. But these commits
>>>>> have been 1-line docstring changes, so I don't see how they could have
>>>>> possibly created the problem.
>>>>>
>>>>> Also, the test passes fine with python 2.7 on my laptop with current master.
>>>>>
>>>>> Can anyone reproduce this failure? Any ideas what might be going on?
>>>>
>>>> I made this:
>>>>
>>>> https://github.com/numpy/numpy/issues/424
>>>>
>>>> It was me who updated the Cython file. It seemed to be working. I've
>>>> added the issue
>>>> to the release TODO.
>>>
>>> Ok, here is how to reproduce the problem:
>>>
>>> 1) install my numpy-vendor vagrant image (32 bit Ubuntu), as directed
>>> in the README:
>>>
>>> https://github.com/certik/numpy-vendor
>>>
>>> 2) run tests, you'll get:
>>>
>>> https://gist.github.com/3625509
>>
>> So the problem was actually introduced much earlier. Most probably it
>> has never worked
>> in 32bit Ubuntu 12.04. I tried for example this old commit:
>>
>> https://github.com/numpy/numpy/commit/3882d65c42acf6d5fff8cc9b3f410bb3e49c8af8
>>
>> and it still fails:
> [...]
>> But in any case, this seems to be a problem with the actual 32bit
>> Ubuntu 12.04 itself. So maybe something in gcc
>> has changed that now triggers the problem.
>
> To be clear, the mismatching value is:
>
>> /home/njs/numpy/.tox/py27/local/lib/python2.7/site-packages/numpy/random/tests/test_random.py(363)test_pareto()
> -> np.testing.assert_array_almost_equal(actual, desired, decimal=15)
> (Pdb) actual[1, 0]
> 52828779.702948704
> (Pdb) desired[1, 0]
> 52828779.702948518
>
> So they do match to 14 decimal points, and it's entirely possible that
> this is just a problem of the test being too stringent in requiring 15
> decimal points of match. Maybe the 32-bit GCC is spilling registers
> differently, tripping the famous x86 idiosyncrasy where register
> spilling triggers rounding. But I'd feel better if someone familiar
> with the pareto code could confirm.

I don't understand this. Isn't assert_almost_equal testing decimals
not significant digits?

As I remember, these tests were added to avoid or signal changes in
the random number generator, and I don't think a digit up or down
makes a difference.

Josef




>
> -n
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion


More information about the NumPy-Discussion mailing list