[SciPy-User] peer review of scientific software

Matthew Brett matthew.brett@gmail....
Thu Jun 6 09:33:17 CDT 2013


Hi,

On Thu, Jun 6, 2013 at 6:49 AM,  <josef.pktd@gmail.com> wrote:
> On Thu, Jun 6, 2013 at 8:57 AM, Matthew Brett <matthew.brett@gmail.com> wrote:
>> Hi,
>>
>> On Thu, Jun 6, 2013 at 1:19 PM,  <josef.pktd@gmail.com> wrote:
>>> On Thu, Jun 6, 2013 at 7:21 AM, Matthew Brett <matthew.brett@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> On Thu, Jun 6, 2013 at 6:23 AM, Jerome Kieffer <Jerome.Kieffer@esrf.fr> wrote:
>>>>> On Wed, 5 Jun 2013 23:08:10 +0100
>>>>> Nathaniel Smith <njs@pobox.com> wrote:
>>>>>
>>>>>> But... have you ever sat down and written tests for a piece of widely
>>>>>> used academic software? (Not LAPACK, but some random large package
>>>>>> that's widely used within a field but doesn't have a comprehensive
>>>>>> test suite of its own.) Everyone I've heard of who's done this
>>>>>> discovers bugs all over the place. Would you personally trip over them
>>>>>> if you didn't test the code? Who knows, maybe not. And probably most
>>>>>> of the rest -- off by one errors here and there, maybe an incorrect
>>>>>> normalizing constant, etc., -- end up not mattering too much. Or maybe
>>>>>> they do. How could you even tell?
>>>>>
>>>>> I found bugs in scipy.ndimage.shift and in scipy.stats.linregress.
>>>>> The first took me ages to be spotted as I was assuming the error was on
>>>>> my side as scipy was seen as a "large library widely used".
>>>>
>>>> Well said.  See also Blake Griffith's current struggles with
>>>> scipy.sparse (last message title "parametric tests, known failures and
>>>> skipped tests").
>>>
>>> As far as I understand these are not BUGs.
>>> These are TDD test failures during development while adding support to
>>> additional dtypes.
>>
>> See for example : https://github.com/scipy/scipy/issues/2542
>>
>> In particular that ticket ends with "Existing tests only tested lil
>> with float data."
>
> you cut off the other part of my statement
>
> But that doesn't mean scipy.sparse didn't work correctly for the
> initial implementation for float matrices.
>
> *float*

Sorry - I think I read your message too quickly.

On the other hand that neatly points out the problem that the user
would be unlikely to guess that sparse would only work correctly for
floats.

Cheers,

Matthew


More information about the SciPy-User mailing list