[Numpy-discussion] consensus (was: NA masks in the next numpy release?)

Matthew Brett matthew.brett@gmail....
Sat Oct 29 20:47:12 CDT 2011


Hi,

On Sat, Oct 29, 2011 at 4:11 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
> Hi,
>
> On Sat, Oct 29, 2011 at 2:59 PM, Charles R Harris
> <charlesr.harris@gmail.com> wrote:
>>
>>
>> On Sat, Oct 29, 2011 at 3:55 PM, Matthew Brett <matthew.brett@gmail.com>
>> wrote:
>>>
>>> Hi,
>>>
>>> On Sat, Oct 29, 2011 at 2:48 PM, Ralf Gommers
>>> <ralf.gommers@googlemail.com> wrote:
>>> >
>>> >
>>> > On Sat, Oct 29, 2011 at 11:36 PM, Matthew Brett
>>> > <matthew.brett@gmail.com>
>>> > wrote:
>>> >>
>>> >> Hi,
>>> >>
>>> >> On Sat, Oct 29, 2011 at 1:48 PM, Matthew Brett
>>> >> <matthew.brett@gmail.com>
>>> >> wrote:
>>> >> > Hi,
>>> >> >
>>> >> > On Sat, Oct 29, 2011 at 1:44 PM, Ralf Gommers
>>> >> > <ralf.gommers@googlemail.com> wrote:
>>> >> >>
>>> >> >>
>>> >> >> On Sat, Oct 29, 2011 at 9:04 PM, Matthew Brett
>>> >> >> <matthew.brett@gmail.com>
>>> >> >> wrote:
>>> >> >>>
>>> >> >>> Hi,
>>> >> >>>
>>> >> >>> On Sat, Oct 29, 2011 at 3:26 AM, Ralf Gommers
>>> >> >>> <ralf.gommers@googlemail.com> wrote:
>>> >> >>> >
>>> >> >>> >
>>> >> >>> > On Sat, Oct 29, 2011 at 1:37 AM, Matthew Brett
>>> >> >>> > <matthew.brett@gmail.com>
>>> >> >>> > wrote:
>>> >> >>> >>
>>> >> >>> >> Hi,
>>> >> >>> >>
>>> >> >>> >> On Fri, Oct 28, 2011 at 4:21 PM, Ralf Gommers
>>> >> >>> >> <ralf.gommers@googlemail.com> wrote:
>>> >> >>> >> >
>>> >> >>> >> >
>>> >> >>> >> > On Sat, Oct 29, 2011 at 12:37 AM, Matthew Brett
>>> >> >>> >> > <matthew.brett@gmail.com>
>>> >> >>> >> > wrote:
>>> >> >>> >> >>
>>> >> >>> >> >> Hi,
>>> >> >>> >> >>
>>> >> >>> >> >> On Fri, Oct 28, 2011 at 3:14 PM, Charles R Harris
>>> >> >>> >> >> <charlesr.harris@gmail.com> wrote:
>>> >> >>> >> >> >>
>>> >> >>> >> >>
>>> >> >>> >> >> No, that's not what Nathaniel and I are saying at all.
>>> >> >>> >> >> Nathaniel
>>> >> >>> >> >> was
>>> >> >>> >> >> pointing to links for projects that care that everyone agrees
>>> >> >>> >> >> before
>>> >> >>> >> >> they go ahead.
>>> >> >>> >> >
>>> >> >>> >> > It looked to me like there was a serious intent to come to an
>>> >> >>> >> > agreement,
>>> >> >>> >> > or
>>> >> >>> >> > at least closer together. The discussion in the summer was
>>> >> >>> >> > going
>>> >> >>> >> > around
>>> >> >>> >> > in
>>> >> >>> >> > circles though, and was too abstract and complex to follow.
>>> >> >>> >> > Therefore
>>> >> >>> >> > Mark's
>>> >> >>> >> > choice of implementing something and then asking for feedback
>>> >> >>> >> > made
>>> >> >>> >> > sense
>>> >> >>> >> > to
>>> >> >>> >> > me.
>>> >> >>> >>
>>> >> >>> >> I should point out that the implementation hasn't - as far as I
>>> >> >>> >> can
>>> >> >>> >> see - changed the discussion.  The discussion was about the API.
>>> >> >>> >>
>>> >> >>> >> Implementations are useful for agreed APIs because they can
>>> >> >>> >> point
>>> >> >>> >> out
>>> >> >>> >> where the API does not make sense or cannot be implemented.  In
>>> >> >>> >> this
>>> >> >>> >> case, the API Mark said he was going to implement - he did
>>> >> >>> >> implement -
>>> >> >>> >> at least as far as I can see.  Again, I'm happy to be corrected.
>>> >> >>> >
>>> >> >>> > Implementations can also help the discussion along, by allowing
>>> >> >>> > people
>>> >> >>> > to
>>> >> >>> > try out some of the proposed changes. It also allows to construct
>>> >> >>> > examples
>>> >> >>> > that show weaknesses, possibly to be solved by an alternative
>>> >> >>> > API.
>>> >> >>> > Maybe
>>> >> >>> > you
>>> >> >>> > can hold the complete history of this topic in your head and
>>> >> >>> > comprehend
>>> >> >>> > it,
>>> >> >>> > but for me it would be very helpful if someone said:
>>> >> >>> > - here's my dataset
>>> >> >>> > - this is what I want to do with it
>>> >> >>> > - this is the best I can do with the current implementation
>>> >> >>> > - here's how API X would allow me to solve this better or simpler
>>> >> >>> > This can be done much better with actual data and an actual
>>> >> >>> > implementation
>>> >> >>> > than with a design proposal. You seem to disagree with this
>>> >> >>> > statement.
>>> >> >>> > That's fine. I would hope though that you recognize that concrete
>>> >> >>> > examples
>>> >> >>> > help people like me, and construct one or two to help us out.
>>> >> >>> That's what use-cases are for in designing APIs.  There are
>>> >> >>> examples
>>> >> >>> of use in the NEP:
>>> >> >>>
>>> >> >>>
>>> >> >>> https://github.com/numpy/numpy/blob/master/doc/neps/missing-data.rst
>>> >> >>>
>>> >> >>> the alterNEP:
>>> >> >>>
>>> >> >>> https://gist.github.com/1056379
>>> >> >>>
>>> >> >>> and my longer email to Travis:
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> http://article.gmane.org/gmane.comp.python.numeric.general/46544/match=ignored
>>> >> >>>
>>> >> >>> Mark has done a nice job of documentation:
>>> >> >>>
>>> >> >>> http://docs.scipy.org/doc/numpy/reference/arrays.maskna.html
>>> >> >>>
>>> >> >>> If you want to understand what the alterNEP case is, I'd suggest
>>> >> >>> the
>>> >> >>> email, just because it's the most recent and I think the
>>> >> >>> terminology
>>> >> >>> is slightly clearer.
>>> >> >>>
>>> >> >>> Doing the same examples on a larger array won't make the point
>>> >> >>> easier
>>> >> >>> to understand.  The discussion is about what the right concepts
>>> >> >>> are,
>>> >> >>> and you can help by looking at the snippets of code in those
>>> >> >>> documents, and deciding for yourself whether you think the current
>>> >> >>> masking / NA implementation seems natural and easy to explain, or
>>> >> >>> rather forced and difficult to explain, and then email back trying
>>> >> >>> to
>>> >> >>> explain your impression (which is not always easy).
>>> >> >>
>>> >> >> If you seriously believe that looking at a few snippets is as
>>> >> >> helpful
>>> >> >> and
>>> >> >> instructive as being able to play around with them in IPython and
>>> >> >> modify
>>> >> >> them, then I guess we won't make progress in this part of the
>>> >> >> discussion.
>>> >> >> You're just telling me to go back and re-read things I'd already
>>> >> >> read.
>>> >> >
>>> >> > The snippets are in ipython or doctest format - aren't they?
>>> >>
>>> >> Oops - 10 minute rule.  Now I see that you mean that you can't
>>> >> experiment with the alternative implementation without working code.
>>> >
>>> > Indeed.
>>> >
>>> >>
>>> >> That's true, but I am hoping that the difference between - say:
>>> >>
>>> >> a[0:2] = np.NA
>>> >>
>>> >> and
>>> >>
>>> >> a.mask[0:2] = False
>>> >>
>>> >> would be easy enough to imagine.
>>> >
>>> > It is in this case. I agree the explicit ``a.mask`` is clearer. This is
>>> > a
>>> > quite specific point that could be improved in the current
>>> > implementation.
>>>
>>> Thanks - this is helpful.
>>>
>>> > It doesn't require ripping everything out.
>>>
>>> Nathaniel wasn't proposing 'ripping everything out' - but backing off
>>> until consensus has been reached.  That's different.    If you think
>>> we should not do that, and you are interested, please say why.
>>> Second - I was proposing that we do indeed keep the code in the
>>> codebase but discuss adaptations that could achieve consensus.
>>>
>>
>> I'm much opposed to ripping the current code out.
>
> You are repeating the loaded phrase 'ripping the current code out' and
> thus making the discussion less sensible and more hostile.
>
>>  It isn't like it is (known
>> to be) buggy, nor has anyone made the case that it isn't a basis on which
>> build other options. It also smacks of gratuitous violence committed by
>> someone yet to make a positive contribution to the project.
>
> This is cheap, rude, and silly.  All I can see from Nathaniel is a
> reasonable, fair attempt to discuss the code.  He proposed backing off
> the code in good faith.   You are emphatically, and, in my view
> childishly, ignoring the substantial points he is making, and
> asserting over and over that he deserves no hearing because he has not
> contributed code.   This is a terribly destructive way to work.  If I
> was a new developer reading this, I would conclude, that I had better
> be damn careful which side I'm on, before I express my opinion,
> otherwise I'm going to be made to feel like I don't exist by the other
> people on the project.  That is miserable, it is silly, and it's the
> wrong way to do business.

I conclude that it's bad to drink this much coffee in an afternoon,
and that the next time I visit my friend's house, I'll take some
decaf.

Sorry Chuck - you're right - this was too personal.   I do disagree
with you, but I was rude here and I am sorry.  I owe you an expensive
drink, as per Ben's excellent suggestion.

See you,

Matthew


More information about the NumPy-Discussion mailing list