[Numpy-discussion] Testing -heads up with #random

[Fernando Perez]

Hi Alan,

I was trying to reuse your #random checker for ipython but kept
running into problems.  Is it working for you in numpy in actual code?
 Because in the entire SVN tree I only see it mentioned here:

maqroll[numpy]> grin #random
./numpy/testing/nosetester.py:
   43 :                     if "#random" in want:
   67 :         # "#random" directive to allow executing a command
while ignoring its
  375 : # try the #random directive on the output line
  379 :     <BadExample object at 0x084D05AC>  #random: may vary on your system
maqroll[numpy]>

I'm asking because I suspect it is NOT working for numpy.  The reason
is some really nasty, silent exception trapping being done by nose.
In nose's loadTestsFromModule,  which you've overridden to include:

                yield NumpyDocTestCase(test,
                                       optionflags=optionflags,
                                       checker=NumpyDoctestOutputChecker())

it's likely that this line can cause an exception (at least it was
doing it for me in ipython, because this class inherits from npd but
tries to directly call __init__ from doctest.DocTestCase).
Unfortunately, nose  will  silently swallow *any* exception there,
simply ignoring your tests and not even telling you what happened.
Very, very annoying.  You can see if you have an exception by doing
something like

            try:
                dt = DocTestCase(test,
                                 optionflags=optionflags,
                                 checker=checker)
            except:
                from IPython import ultraTB
                ultraTB.AutoFormattedTB()()
            yield dt

to force a traceback printing.

Anyway, I mention this because I just wasted a good chunk of time
fighting this one for ipython, where I need the #random functionality.
 It seems it's not used in numpy yet, but I imagine it will soon, and
I figured I'd save you some time.

Cheers,

f