[Numpy-discussion] Many failing doctests - release blocker? Enable for default test runs?
Mon Dec 24 12:08:03 CST 2012
On Sun, Dec 23, 2012 at 6:54 PM, Ondřej Čertík <email@example.com> wrote:
> Hi Matthew,
> On Sat, Dec 22, 2012 at 9:40 PM, Matthew Brett <firstname.lastname@example.org> wrote:
>> I noticed that enabling the doctests on the 1.7.x maintenance branch
>> caused lots and lots of doctest failures.
>> (np-devel)[mb312@blair ~/dev_trees]$ python -c 'import numpy as np;
>> Running unit tests and doctests for numpy
>> NumPy version 1.7.0rc1.dev-1e8fcdf
>> NumPy is installed in
>> Python version 2.6.6 (r266:84374, Aug 31 2010, 11:00:51) [GCC 4.0.1
>> (Apple Inc. build 5493)]
>> nose version 1.1.2
>> Ran 3839 tests in 59.928s
>> FAILED (KNOWNFAIL=4, SKIP=4, errors=23, failures=175)
>> The doctests also throw up somewhere round 10 matplotlib plots, so
>> presumably those would fail as well on a machine without a display
>> without forcing the import of an 'Agg' backend or similar.
>> I have never checked the doctests on Python 3. Has anyone run those recently?
>> For the projects I work on most, we enable doctests for the default
>> test run - as in 'doctests=True' by default in the numpy testing
>> machinery. Do ya'll see any disadvantage in doing that for numpy?
>> In case someone gets to this before I do, we've also got some logic
>> for doing conditional skips of doctests when optional packages are not
>> available such as matplotlib, inspired by something similar in
>> If Christmas allows I'll send a pull request with something like that
>> in the next few days.
> Thanks for pointing this out. I think in the long term, we should definitely
> run doctests as part of the test suite on Travis-CI. Because what use
> is a doctest if it doesn't work?
> Matthew, do you know if doctests fail for the 1.6 release as well?
FAILED (KNOWNFAIL=5, SKIP=3, errors=43, failures=167)
On Python 3.2, current 1.7.x maintenance:
FAILED (KNOWNFAIL=5, SKIP=4, errors=24, failures=211)
The last time I looked I had the impression that we were not doing
2to3 conversion on doctests, but that was a while ago.
More information about the NumPy-Discussion