[Numpy-discussion] What is consensus anyway

Travis Oliphant travis@continuum...
Tue Apr 24 23:01:45 CDT 2012

On Apr 24, 2012, at 9:41 PM, Matthew Brett wrote:

> Hi,
> On Tue, Apr 24, 2012 at 6:12 PM, Charles R Harris
> <charlesr.harris@gmail.com> wrote:
>> On Tue, Apr 24, 2012 at 6:56 PM, Nathaniel Smith <njs@pobox.com> wrote:
>>> On Tue, Apr 24, 2012 at 2:14 PM, Charles R Harris
>>> <charlesr.harris@gmail.com> wrote:
>>>> On Mon, Apr 23, 2012 at 11:35 PM, Fernando Perez <fperez.net@gmail.com>
>>>> wrote:
>>>>> On Mon, Apr 23, 2012 at 8:49 PM, Stéfan van der Walt <stefan@sun.ac.za>
>>>>> wrote:
>>>>>> If you are referring to the traditional concept of a fork, and not to
>>>>>> the type we frequently make on GitHub, then I'm surprised that no one
>>>>>> has objected already.  What would a fork solve? To paraphrase the
>>>>>> regexp saying: after forking, we'll simply have two problems.
>>>>> I concur with you here: github 'forks', yes, as many as possible!
>>>>> Hopefully every one of those will produce one or more PRs :)  But a
>>>>> fork in the sense of a divergent parallel project?  I think that would
>>>>> only be indicative of a complete failure to find a way to make
>>>>> progress here, and I doubt we're anywhere near that state.
>>>>> That forks are *possible* is indeed a valuable and important option in
>>>>> open source software, because it means that a truly dysfunctional
>>>>> original project team/direction can't hold a community hostage
>>>>> forever.  But that doesn't mean that full-blown forks should be
>>>>> considered lightly, as they also carry enormous costs.
>>>>> I see absolutely nothing in the current scenario to even remotely
>>>>> consider that a full-blown fork would be a good idea, and I hope I'm
>>>>> right.  It seems to me we're making progress on problems that led to
>>>>> real difficulties last year, but from multiple parties I see signs
>>>>> that give me reason to be optimistic that the project is getting
>>>>> better, not worse.
>>>> We certainly aren't there at the moment, but I can see us heading that
>>>> way.
>>>> But let's back up a bit. Numpy 1.6.0 came out just about 1 year ago.
>>>> Since
>>>> then datetime, NA, polynomial work, and various other enhancements have
>>>> gone
>>>> in along with some 280 bug fixes. The major technical problem blocking a
>>>> 1.7
>>>> release is getting datetime working reliably on windows. So I think that
>>>> is
>>>> where the short term effort needs to be. Meanwhile, we are spending
>>>> effort
>>>> to get out a 1.6.2 just so people can work with a stable version with
>>>> some
>>>> of the bug fixes, and potentially we will spend more time and effort to
>>>> pull
>>>> out the NA code. In the future there may be a transition to C++ and
>>>> eventually a break with the current ABI. Or not.
>>>> There are at least two motivations that get folks to write code for open
>>>> source projects, scratching an itch and money. Money hasn't been a big
>>>> part
>>>> of the Numpy picture so far, so that leaves scratching an itch. One of
>>>> the
>>>> attractions of Numpy is that it is a small project, BSD licensed, and
>>>> not
>>>> overburdened with governance and process. This makes scratching an itch
>>>> not
>>>> as difficult as it would be in a large project. If Numpy remains a small
>>>> project but acquires the encumbrances of a big project much of that
>>>> attraction will be lost. Momentum and direction also attracts people,
>>>> but
>>>> numpy is stalled at the moment as the whole NA thing circles around once
>>>> again.
>>> I don't think we need a fork, or to start maintaining separate stable
>>> and unstable trees, or any of the other complicated process changes
>>> that have been suggested. There are tons of projects that routinely
>>> make much bigger changes than we're talking about, and they do it
>>> without needing that kind of overhead. I know that these suggestions
>>> are all made in good faith, but they remind me of a line from that
>>> Apache page I linked earlier: "People tend to avoid conflict and
>>> thrash around looking for something to substitute - somebody in
>>> charge, a rule, a process, stagnation. None of these tend to be very
>>> good substitutes for doing the hard work of resolving the conflict."
>>> I also think if you talk to potential contributors, you'll find that
>>> clear, simple processes and a history of respecting everyone's input
>>> are much more attractive than a no-rules free-for-all. Good
>>> engineering practices are not an "encumbrance". Resolving conflicts
>>> before merging is a good engineering practice.
>>> What happened with the NA discussion is this:
>>>  - There was substantial disagreement about whether NEP-style masks,
>>> or indeed, focusing on a mask-based implementation *at all*, was the
>>> best way forward.
>>>  - There was also a perceived time constraint, that we had to either
>>> implement something immediately while Mark was there, or have nothing.
>>> So in the end, the latter concern outweighed the former, the
>>> discussion was cut off, and Mark's best guess at an API was merged
>>> into master. I totally understand how this decision made sense at the
>>> time, but the result is what we see now: it's left numpy stalled,
>>> rifts on the mailing list, boring discussions about process, and still
>>> no agreement about whether NEP-style masks will actually solve our
>>> users' problems.
>>> Getting past this isn't *complicated* -- it's just "hard work".
>> I admit to a certain curiosity about your own involvement in FOSS projects,
>> and I know I'm not alone in this. Google shows several years of discussion
>> on Monotone, but I have no idea what your contributions were outside of
>> documentation. Monotone has sort of died, but that is the luck of the draw.
>> Would you care to comment on the success of the process in that project and
>> what went right or wrong? The other thing I see your name attached to is
>> xpra, which gets good reviews, but that was a personal project and forked
>> after you found more interesting things to work on.
> I'm sorry, but I think this is really not OK. This is an ad-hominem attack [1].
> I'm not going to join in by justifying Nathaniel's expertise, because
> I do not think he should have to do that.  I just think it is wrong to
> ask people to justify their expertise when we should be discussing the
> ideas.
> If I was a new subscriber to this list, and saw that my right to speak
> was likely to be audited in public if I disagree, I would be very
> reluctant to differ from the majority.  I do not think that is what we
> want.

I agree.   I really hope that was not the intent.  I'm willing to believe that Chuck had only curiosity has his motivator, or perhaps is really trying to understand Nathaniel's experience so that he can contrast it with his own and make his own mind up about what the best ways to participate in open source development are. 

However, when I read this I saw how this could be taken in the wrong way and thought "This is not the right line of questioning"    I would make a petition that we try and please just focus on the technical discussion. 

> I think his analysis of the issues in the masked array API are also
> reasonable, well-thought out, and well-argued.  I think we should
> address his arguments.  I think we're getting stuck precisely because
> you will not do that.

While I agree with your general assessment of this particular email, I don't think that is entirely fair. I'm sure you recognize that Chuck is addressing technical issues and providing possible solutions in many of his emails.   It's just that this particular line of questioning seems out of place and not relevant.   I do hope this does not continue.   


More information about the NumPy-Discussion mailing list