[SciPy-dev] the state of scipy unit tests

David Cournapeau david@ar.media.kyoto-u.ac...
Mon Nov 24 01:09:41 CST 2008


Nathan Bell wrote:
> In the past, I would always run 'nosetests scipy' before committing
> changes to SVN.  Due to the current state of the unit tests, I don't
> anymore, and I suspect I'm not alone.
>
> Here are the main offenders on my system:
>
> scipy.stats.
> I appreciate the fact that rigorous testing on this module takes time,
> but 4 minutes on a 2.4GHz Core 2 system is unreasonable.  IMO 20
> seconds is a reasonable upper bound.  Essential tests that don't meet
> this time constraint should be filtered out of the default test suite.
>   

I don't agree much on that reasoning. Test are useful; the more run by
default, the better; tests which are not run by default are nearly
useless IMO, since not many people would run tests with options; since
there are ways to restrict tests to a meaningful subset (per
subpackage), I think this is enough; if some tests can be run faster,
then ok, but not if it requires to lose some test coverage.

Why does the test time matter so much to you ?

> scipy.weave
> Takes 2.5 minutes and litters my screen with a few hundred lines like,
> /tmp/tmpcRI5WR/sc_3a5c7ad3ac45a98d03cd9168232f7d8f1.cpp:618: warning:
> deprecated conversion from
> and dumps a bunch of .cpp and .so files in the current working directory.
>
> Some minor offenders:
>
> scipy.interpolate
> Emits several UserWarnings (should be filtered out with warnings.warnfilter).
>
> scipy.io
> Several DeprecationWarnings
>
> scipy.lib
> A dozen lines like "zcopy:n=3"
>
> scipy.linalg
> Outputs ATLAS info and a dozen lines like "zcopy:n=3".
>   

Yes, those should be cleaned (except maybe DeprecationWarnings - if the
deprecated functions are the one being tested: I am not sure what we
should do in that case). But that's a lot of grunt work ; particularly
scipy.lib, which should be removed IMHO (as for now, it is mostly
redundant with scipy.linalg, except for some unit tests which would be
useful to put into scipy.linalg).

> I'd like to be able to run the entire battery of tests in about a
> minute with minimal unnecessary output.
>   

I hear you, I would like the whole build + test process for scipy to be
faster too :) If 4 minutes sounds long, what about build + test on
windows, which takes at least 20 minutes (to multiply by three when I
build the superpack - and the process can't even be controlled remotely) !

David



More information about the Scipy-dev mailing list