[Numpy-discussion] More pending test framework changes (please give feedback)
Mon Jun 30 12:54:59 CDT 2008
I can commit any/all of the following changes as long as they are
agreeable to everybody:
1. All doctests in NumPy will have the numpy module available in their
execution context as "np".
2. Turn on the normalized whitespace option for all doctests. Having
a doctest fail just because there's a space after your result seems
like an unnecessary hassle for documenters.
3. Output will be ignored for each doctest expected output line that
contains "#random". I figured this can serve both as an ignore flag
and indication to the reader that the listed output may differ from
what they see if they execute the associated command. So you would be
able to do:
0.1234567890 #random: output may differ on your system
And have the example executed but not cause a failure. You could also
use this to ignore the <SomeObject at 0x1234ABCD> output from plot
methods as well.
If these get committed, I'll also revert doctests I changed earlier so
that they use the provided "np" instead of importing it, and fix any
currently failing doctests that can be addressed with the new
behavior. I've already got a batch of these changes made locally just
to try out the new features, but I'll wait to see if adding the
features causes any troubles on the buildbots first.
There are a couple of options that I haven't got patches for yet:
4. Doctest commands starting with "plt." or "plot." will automatically
have their output ignored.
5. Add a decorator that causes all doctest output for that
function/method's docstring to be ignored. (This is for cases with
many examples, all of which should be executed but not compared, so
you don't have to clutter up each output line with #random or
If there is still a big need for these, I'll work on them. As an
alternative to 3,4 and 5, you could use the ELLIPSIS directive to
ignore the varying bits of the output, but you'd still have to put the
directive on each command line where you needed it. Unless I turn on
the ellipsis feature for all doctests, that is; it's easy to do, and
it doesn't cause me any obvious problems locally. I don't know
whether that's something that's generally a good idea, though.
More information about the Numpy-discussion