[SciPy-dev] the state of scipy unit tests

Robert Kern robert.kern@gmail....
Mon Nov 24 01:44:04 CST 2008


On Mon, Nov 24, 2008 at 01:09, David Cournapeau
<david@ar.media.kyoto-u.ac.jp> wrote:
> Nathan Bell wrote:
>> In the past, I would always run 'nosetests scipy' before committing
>> changes to SVN.  Due to the current state of the unit tests, I don't
>> anymore, and I suspect I'm not alone.
>>
>> Here are the main offenders on my system:
>>
>> scipy.stats.
>> I appreciate the fact that rigorous testing on this module takes time,
>> but 4 minutes on a 2.4GHz Core 2 system is unreasonable.  IMO 20
>> seconds is a reasonable upper bound.  Essential tests that don't meet
>> this time constraint should be filtered out of the default test suite.
>
> I don't agree much on that reasoning. Test are useful; the more run by
> default, the better; tests which are not run by default are nearly
> useless IMO, since not many people would run tests with options; since
> there are ways to restrict tests to a meaningful subset (per
> subpackage), I think this is enough; if some tests can be run faster,
> then ok, but not if it requires to lose some test coverage.
>
> Why does the test time matter so much to you ?

You want to be able to run the main automated test suite every time
before you do a check in, and more frequently while you are working on
something, so that you make sure you didn't break things you weren't
working on. This is a fairly well-accepted principle of testing.

No one is suggesting that tests should be deleted, just that they
might be moved (or marked) out of the main test suite. Multiple test
suites for different purposes and constraints is far from uncommon.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


More information about the Scipy-dev mailing list