[Numpy-discussion] Numpy governance update

Christopher Jordan-Squire cjordan1@uw....
Thu Feb 16 01:37:31 CST 2012


On Wed, Feb 15, 2012 at 5:26 PM, Mark Wiebe <mwwiebe@gmail.com> wrote:
> On Wed, Feb 15, 2012 at 4:57 PM, <josef.pktd@gmail.com> wrote:
>>
>> On Wed, Feb 15, 2012 at 7:27 PM, Dag Sverre Seljebotn
>> <d.s.seljebotn@astro.uio.no> wrote:
>> > On 02/15/2012 02:24 PM, Mark Wiebe wrote:
>> >> On Wed, Feb 15, 2012 at 1:36 PM, Matthew Brett <matthew.brett@gmail.com
>> >> <mailto:matthew.brett@gmail.com>> wrote:
>> >>
>> >>     Hi,
>> >>
>> >>     On Wed, Feb 15, 2012 at 12:55 PM, Mark Wiebe <mwwiebe@gmail.com
>> >>     <mailto:mwwiebe@gmail.com>> wrote:
>> >>      > On Wed, Feb 15, 2012 at 12:09 PM, Matthew Brett
>> >>     <matthew.brett@gmail.com <mailto:matthew.brett@gmail.com>>
>> >>      > wrote:
>> >>      >>
>> >>      >> Hi,
>> >>      >>
>> >>      >> On Wed, Feb 15, 2012 at 11:46 AM, Benjamin Root
>> >> <ben.root@ou.edu
>> >>     <mailto:ben.root@ou.edu>> wrote:
>> >>      >> >
>> >>      >> >
>> >>      >> > On Wed, Feb 15, 2012 at 1:32 PM, Alan G Isaac
>> >>     <alan.isaac@gmail.com <mailto:alan.isaac@gmail.com>>
>> >>      >> > wrote:
>> >>      >> >> Can you provide an example where a more formal
>> >>      >> >> governance structure for NumPy would have meant
>> >>      >> >> more or better code development? (Please do not
>> >>      >> >> suggest the NA discussion!)
>> >>      >> >>
>> >>      >> >
>> >>      >> > Why not the NA discussion?  Would we really want to have that
>> >>     happen
>> >>      >> > again?
>> >>      >> > Note that it still isn't fully resolved and progress still
>> >>     needs to be
>> >>      >> > made
>> >>      >> > (I think the last thread did an excellent job of fleshing out
>> >>     the ideas,
>> >>      >> > but
>> >>      >> > it became too much to digest.  We may need to have someone go
>> >>     through
>> >>      >> > the
>> >>      >> > information, reduce it down and make one last push to bring
>> >> it
>> >>     to a
>> >>      >> > conclusion).  The NA discussion is the perfect example where
>> >> a
>> >>      >> > governance
>> >>      >> > structure would help resolve disputes.
>> >>      >>
>> >>      >> Yes, that was the most obvious example. I don't know about you,
>> >>     but I
>> >>      >> can't see any sign of that one being resolved.
>> >>      >>
>> >>      >> The other obvious example was the dispute about ABI breakage
>> >> for
>> >>     numpy
>> >>      >> 1.5.0 where I believe Travis did invoke some sort of committee
>> >> to
>> >>      >> vote, but (Travis can correct me if I'm wrong), the committee
>> >> was
>> >>      >> named ad-hoc and contacted off-list.
>> >>      >>
>> >>      >> >
>> >>      >> >>
>> >>      >> >> Can you provide an example of what you might
>> >>      >> >> envision as a "more formal governance structure"?
>> >>      >> >> (I assume that any such structure will not put people
>> >>      >> >> who are not core contributors to NumPy in a position
>> >>      >> >> to tell core contributors what to spend their time on.)
>> >>      >> >>
>> >>      >> >> Early last December, Chuck Harris estimated that three
>> >>      >> >> people were active NumPy developers.  I liked the idea of
>> >>      >> >> creating a "board" of these 3 and a rule that says any
>> >>      >> >> active developer can request to join the board, that
>> >>      >> >> additions are determined by majority vote of the existing
>> >>      >> >> board, and  that having the board both small and odd
>> >>      >> >> numbered is a priority.  I also suggested inviting to this
>> >>      >> >> board a developer or two from important projects that are
>> >>      >> >> very NumPy dependent (e.g., Matplotlib).
>> >>      >> >>
>> >>      >> >> I still like this idea.  Would it fully satisfy you?
>> >>      >> >>
>> >>      >> >
>> >>      >> > I actually like that idea.  Matthew, is this along the lines
>> >>     of what you
>> >>      >> > were thinking?
>> >>      >>
>> >>      >> Honestly it would make me very happy if the discussion moved to
>> >> what
>> >>      >> form the governance should take.  I would have thought that 3
>> >>     was too
>> >>      >> small a number.
>> >>      >
>> >>      >
>> >>      > One thing to note about this point is that during the NA
>> >>     discussion, the
>> >>      > only people doing active C-level development were Charles and
>> >> me.
>> >>     I suspect
>> >>      > a discussion about how to recruit more people into that group
>> >>     might be more
>> >>      > important than governance at this point in time.
>> >>
>> >>     Mark - a) thanks for replying, it's good to hear your voice and b)
>> >> I
>> >>     don't think there's any competition between the discussion about
>> >>     governance and the need to recruit more people into the group who
>> >>     understand the C code.
>> >>
>> >>
>> >> There hasn't really been any discussion about recruiting developers to
>> >> compete with the governance topic, now we can let the topics compete.
>> >> :)
>> >>
>> >> Some of the mechanisms which will help are already being set in motion
>> >> through the discussion about better infrastructure support like bug
>> >> trackers and continuous integration. The forthcoming roadmap discussion
>> >> Travis alluded to, where we will propose a roadmap for review by the
>> >> numpy user community, will include many more such points.
>> >>
>> >>     Remember we are deciding here between governance - of a form to be
>> >>     decided - and no governance - which I think is the current
>> >> situation.
>> >>     I know your desire is to see more people contributing to the C
>> >> code.
>> >>     It would help a lot if you could say what you think the barriers
>> >> are,
>> >>     how they could be lowered, and the risks that you see as a result
>> >> of
>> >>     the numpy C expertise moving essentially into one company.  Then we
>> >>     can formulate some governance that would help lower those barriers
>> >> and
>> >>     reduce those risks.
>> >>
>> >>
>> >> There certainly is governance now, it's just informal. It's a
>> >> combination of how the design discussions are carried out, how pull
>> >> requests occur, and who has commit rights.
>> >
>> > +1
>> >
>> > If non-contributing users came along on the Cython list demanding that
>> > we set up a system to select non-developers along on a board that would
>> > have discussions in order to veto pull requests, I don't know whether
>> > we'd ignore it or ridicule it or try to show some patience, but we
>> > certainly wouldn't take it seriously.
>> >
>> > It's obvious that one should try for consensus as long as possible,
>> > including listening to users. But in the very end, when agreement can't
>> > be reached by other means, the developers are the one making the calls.
>> > (This is simply a consequence that they are the only ones who can
>> > credibly threaten to fork the project.)
>> >
>> > Sure, structures that includes users in the process could be useful...
>> > but, if the devs are fine with the current situation (and I don't see
>> > Mark or Charles complaining), then I honestly think it is quite rude to
>> > not let the matter drop after the first ten posts or so.
>> >
>> > Making things the way one wants it and scratching *ones own* itch is THE
>> > engine of open source development (whether one is putting in spare time
>> > or monetary funding). Trying to work against that with artificial
>> > structures doesn't sound wise for a project with as few devs as NumPy...
>>
>> I don't think you can restrict the Numpy developer or contributor
>> group just to the developers that work on the C core like Charles and
>> Mark, and others over the years I have been following it ( Pauli and
>> David, ...).
>> There is a large part of non C numpy, Pierre for example, and Joe
>> Harrington put money and a lot of effort into bringing the
>> documentation into the current state, the documentation was mostly a
>> community effort.
>
>
> This is very true, at the moment the number of people doing feature-work
> within numpy purely in Python is similarly small and sporadic. Here's a
> current example:
>
> https://github.com/numpy/numpy/pull/198
>
> Having such a small development core is one of the reasons it often takes a
> while for such pull requests to get reviewed by someone, and a situation
> Continuum and anyone else with resources to contribute can help improve. One
> thing that's clear to me is that the current documentation on how to
> contribute code, documentation, and other help to NumPy is lacking, and this
> is something that needs improvement.
>
> An example I really like is LibreOffice's "get involved" page.
>
> http://www.libreoffice.org/get-involved/
>
> Producing something similar for NumPy will take some work, but I believe
> it's needed.
>

 +1 to Mark, Perry, Dag.

On the topic of 'nifty webpages other projects have that I wish
numpy/scipy had', eigen (an open-source C++ linear algebra library)
has a nice To Do list that gives concrete, approachable things new
developers could add to:
http://eigen.tuxfamily.org/index.php?title=Todo.

'Course, that would mean people agreeing on what there was to do....

-Chris JS


> Cheers,
> Mark
>
>>
>>
>> Of course I only ever contributed to scipy, except of two or three
>> bugfixes in numpy.random, but I still care about the direction numpy
>> is going, as do developers of the "SciPy" community which crucially
>> rely on numpy.
>>
>> Josef
>>
>>
>> >
>> > Dag
>> > _______________________________________________
>> > NumPy-Discussion mailing list
>> > NumPy-Discussion@scipy.org
>> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


More information about the NumPy-Discussion mailing list