[Numpy-discussion] ticket 788: possible blocker

Stéfan van der Walt stefan@sun.ac...
Tue May 13 17:40:39 CDT 2008


2008/5/13 Travis E. Oliphant <oliphant@enthought.com>:
> Robert Kern wrote:
>  > On Tue, May 13, 2008 at 11:12 AM, Travis E. Oliphant
>  > <oliphant@enthought.com> wrote:
>  >
>  >
>  >>  Besides,  having a "test-per-checkin" is not the proper mapping in my
>  >>  mind.   I'd rather see whole check-ins devoted to testing large pieces
>  >>  of code rather than spend all unit-test foo on a rigid policy of
>  >>  "regression" testing each check-in.
>  >>
>  >
>  > Stéfan is proposing "test-per-bugfix", not "test-per-checkin". That is
>  > eminently feasible. You need to do some kind of testing to be sure
>  > that you actually fixed the problem. It is simply *not* *that* *hard*
>  > to write that in unit test form.
>  >
>  That is not true.  You *don't* need to do testing to be sure you
>  actually fixed the problem in some cases....  Looking at the code is
>  enough.   Like the case we are talking about.

That is where we disagree: looking at the code is simply not enough.
It is fine if you know where to look, and if you look exactly there
every day -- but who does that?  That is precisely why we have tests
-- to automate this inspection.

You know this code base extremely well, but what may seem obvious to
you won't be to another developer a year down the line, and that
person needs your assistance now already.  When a person works on a
ticket, he become intimately familiar with its contents.  That means
that, where it would take that person a couple of minutes to write a
test, it would take another developer many more, because the other
developer first needs to gain all the relevant background knowledge.

For the case in question, Eric provided test data.  In order to fix
the bug, it was needed to run that code; so whether it was pasted it
into the test suite or whether it was run from a terminal wouldn't
have made much of a difference in developer time.  It would, however,
have provided us with a guarantee that we conquered that bug once, and
more importantly, for all.

> Let's go write unit
> tests and stop harassing people who are fixing bugs.

It is not my intention to harass you or anyone else (and I apologise
if it came across that way) for fixing bugs. In fact, I am
(incredibly) grateful for these efforts.  The reasons I am unhappy is
because

a) I have no guarantee that any bug was fixed and
b) I could be stupid enough to break that functionality tonight and
no-one would be any the wiser

> NumPy would not exist if I had followed the process you seem to want
> now.

I'm not sure I agree.  The tests were written anyway -- no-one codes
without trying their code.  I'm simply arguing that those trials
should end up somewhere useful.  If Numeric was developed from a test
driven perspective we could, for example, have avoided breaking *any*
of the maskedarray functionality in the merge.  Similarly, we could
have caught some of these memory errors that keep popping up a long
time ago.

I remain of the opinion that the benefits of decent testing completely
overwhelms the tiny effort it takes to put into place.

Regards
Stéfan


More information about the Numpy-discussion mailing list