[Numpy-discussion] Many failing doctests - release blocker? Enable for default test runs?
Sun Dec 23 14:53:21 CST 2012
On Sun, Dec 23, 2012 at 7:54 PM, Ondřej Čertík <email@example.com>wrote:
> Hi Matthew,
> On Sat, Dec 22, 2012 at 9:40 PM, Matthew Brett <firstname.lastname@example.org>
> > Hi,
> > I noticed that enabling the doctests on the 1.7.x maintenance branch
> > caused lots and lots of doctest failures.
> > (np-devel)[mb312@blair ~/dev_trees]$ python -c 'import numpy as np;
> > np.test(doctests=True)'
> > 1.7.0rc1.dev-1e8fcdf
> > Running unit tests and doctests for numpy
> > NumPy version 1.7.0rc1.dev-1e8fcdf
> > NumPy is installed in
> > /Users/mb312/.virtualenvs/np-devel/lib/python2.6/site-packages/numpy
> > Python version 2.6.6 (r266:84374, Aug 31 2010, 11:00:51) [GCC 4.0.1
> > (Apple Inc. build 5493)]
> > nose version 1.1.2
> > ...
> > Ran 3839 tests in 59.928s
> > FAILED (KNOWNFAIL=4, SKIP=4, errors=23, failures=175)
> > The doctests also throw up somewhere round 10 matplotlib plots, so
> > presumably those would fail as well on a machine without a display
> > without forcing the import of an 'Agg' backend or similar.
> > I have never checked the doctests on Python 3. Has anyone run those
> > For the projects I work on most, we enable doctests for the default
> > test run - as in 'doctests=True' by default in the numpy testing
> > machinery. Do ya'll see any disadvantage in doing that for numpy?
Yes, I do. The doctest framework and reproducibility of reprs across Python
versions and platforms are too poor to do this. And failing tests give new
users a bad impression of the quality of numpy.
I'm +1 on enabling doctests on Travis for one Python version (2.7 probably)
in order to reduce the number of out-of-date examples, -1 on default
> > In case someone gets to this before I do, we've also got some logic
> > for doing conditional skips of doctests when optional packages are not
> > available such as matplotlib, inspired by something similar in
> > IPython:
> > https://github.com/nipy/nipy/blob/master/nipy/testing/doctester.py#L193
> > If Christmas allows I'll send a pull request with something like that
> > in the next few days.
> Thanks for pointing this out. I think in the long term, we should
> run doctests as part of the test suite on Travis-CI. Because what use
> is a doctest if it doesn't work?
Since a "doctest" is an example and not a test, still quite useful.
> Matthew, do you know if doctests fail for the 1.6 release as well?
> I am swamped with other bugs for the 1.7 release and since I assume
> they also fail for 1.6, I want to get the release out as soon as we fix our
> current issues.
Agreed that this shouldn't be a release blocker.
> However, I think it's a good idea to run doctests automatically on Travis,
> once they are all fixed.
> NumPy-Discussion mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the NumPy-Discussion