[SciPy-User] peer review of scientific software

Matthew Brett matthew.brett@gmail....
Thu Jun 6 07:57:28 CDT 2013


Hi,

On Thu, Jun 6, 2013 at 1:19 PM,  <josef.pktd@gmail.com> wrote:
> On Thu, Jun 6, 2013 at 7:21 AM, Matthew Brett <matthew.brett@gmail.com> wrote:
>> Hi,
>>
>> On Thu, Jun 6, 2013 at 6:23 AM, Jerome Kieffer <Jerome.Kieffer@esrf.fr> wrote:
>>> On Wed, 5 Jun 2013 23:08:10 +0100
>>> Nathaniel Smith <njs@pobox.com> wrote:
>>>
>>>> But... have you ever sat down and written tests for a piece of widely
>>>> used academic software? (Not LAPACK, but some random large package
>>>> that's widely used within a field but doesn't have a comprehensive
>>>> test suite of its own.) Everyone I've heard of who's done this
>>>> discovers bugs all over the place. Would you personally trip over them
>>>> if you didn't test the code? Who knows, maybe not. And probably most
>>>> of the rest -- off by one errors here and there, maybe an incorrect
>>>> normalizing constant, etc., -- end up not mattering too much. Or maybe
>>>> they do. How could you even tell?
>>>
>>> I found bugs in scipy.ndimage.shift and in scipy.stats.linregress.
>>> The first took me ages to be spotted as I was assuming the error was on
>>> my side as scipy was seen as a "large library widely used".
>>
>> Well said.  See also Blake Griffith's current struggles with
>> scipy.sparse (last message title "parametric tests, known failures and
>> skipped tests").
>
> As far as I understand these are not BUGs.
> These are TDD test failures during development while adding support to
> additional dtypes.

See for example : https://github.com/scipy/scipy/issues/2542

In particular that ticket ends with "Existing tests only tested lil
with float data."

Not that this is surprising - I learned to test the hell out of
everything by finding how often I wrote broken code myself.

Cheers,

Matthew


More information about the SciPy-User mailing list