[SciPy-dev] Scipy workflow (and not tools).

Bruce Southey bsouthey@gmail....
Wed Feb 25 12:16:51 CST 2009

Travis E. Oliphant wrote:
> Charles R Harris wrote:
>> I don't think there are enough eyes at this point for a strict review 
>> policy. How many of the current packages have any maintainer? Who was 
>> maintaining the stats package before Josef got involved? How many 
>> folks besides Robert could look over the changes usefully? How many 
>> folks looked over Travis' recent addition to optimize?  Who is working 
>> on the interpolation package?
>> I think at this point we would be better off trying to recruit at 
>> least one person to "own" each package. For new packages that is 
>> usually the person who committed it but we also need ownership of 
>> older packages. Someone with a personal stake in a package is likely 
>> to do more for quality assurance at this point than any amount of 
>> required review.
> Yes,  my feelings exactly.   Quality goes up when people who have a 
> personal stake or attachment to the code are engaged.   How do we get 
> more of this to happen?   Formal review processes can actually have at 
> least some negative impact in getting people engaged.     Let's make a 
> tweak here and a tweak there.   Right now, I'm of the opinion that 
> whatever makes the *workflow* of people like David, Pauli, Jarrod, 
> Robert K, Robert C, Nathan, Matthew, Charles, Anne, Andrew, Gael, and 
> Stefan (and others big contributors I may have missed) easier, I'm 
> totally in favor of.    If that is a DVCS and/or something different 
> than Trac, then let's do that. 
> It sounds like we are making steps in that direction which is excellent.
Really based on the discussion (including the latter comments), it 
appears to me that this discussion has moved towards what sort of 
developmental structure should scipy be using with a DVCS.

I viewed much of the discussion following what sort of happens with 
Linux kernel development since they adopted DVCS starting with 
Bitkeeper. Jonathon Corbet has an interesting article on this 
http://ldn.linuxfoundation.org/book/how-participate-linux-community . 
Essentially there are sub-maintainer trees that feed into the testing 
tree (-mm), the staging tree (where patches are applied against that 
should minimize tree divergence) and hopefully Linus's tree.  During 
that process is informal code review for at least bug fixes as new or 
major features still have a problem with code review. In some aspects, 
the man-power restriction with the Linux kernel development has been 
removed because code no longer has to flow through a single node. So 
this allows a user to get easily get code not only from these trees but 
also other developers.

So scipy could do something similar where the use of DVCS which would 
hopefully this would reduce the burden on people like Robert and you.

I do not see a real need at this time to say you 'own' that module and 
you must 'control' the development of it. I would suspect that 
'ownership' of scipy components will naturally develop over time and, 
thus, should not be forced upon anyone.

If sub-trees were created it would permit a sharing of incomplete code 
so that the burden of developing appropriate tests, writing 
documentation and testing can be distributed to interested parties. This 
would also foster mentoring and getting hands dirty in a positive way.

>> I don't have a problem with folks complaining about missing tests, 
>> etc., but I worry that if we put too many review steps into the 
>> submission path there won't be enough people to make it work.
> This is exactly the way I feel....  I don't want to imply at all that we 
> shouldn't be bugging each other about documentation and testing.  I 
> personally welcome any reminders in that direction.  I am just worried 
> about whether or not we are really solving the real problems that make 
> it hard to contribute by instituting policy rather than providing 
> examples of code to model.
 From this it appears that in order to get code into scipy then you have 
to have all this documentation and tests. But really my concern is 
strict requirements of tests and documentation is that we will get 
minimal tests and inferior documentation or nothing at all. Rather I 
hope that the burden can be shifted from one person to a group of people 
then perhaps we can get more extensive tests and documentation as well 
as people actually testing the code on different systems with hopefully 
realistic situations. So at least there is some fix in some tree for a 
problem and eventually the rest of it will follow by the time everything 
is ready for mainline inclusion.


More information about the Scipy-dev mailing list