[SciPy-dev] Technology Previews and Sphinx Docstring Annotations

Damian Eads eads@soe.ucsc....
Fri Nov 7 00:41:43 CST 2008

On Thu, Nov 6, 2008 at 9:24 PM, Charles R Harris
<charlesr.harris@gmail.com> wrote:
> On Tue, Nov 4, 2008 at 6:25 PM, David Cournapeau <cournape@gmail.com> wrote:
>> On Wed, Nov 5, 2008 at 7:34 AM, Anne Archibald
>> <aarchiba@physics.mcgill.ca> wrote:
>> > 2008/11/4 Travis E. Oliphant <oliphant@enthought.com>:
>> >> Jarrod Millman wrote:
>> >>> I absolutely agree with the ideas presented about scikits and look
>> >>> forward to seeing the numerous scikits improvements.  I feel that I
>> >>> have gotten into a discussion where the counter argument to what I am
>> >>> proposing is something I strongly support.  I also feel that the
>> >>> counterargument doesn't directly address my concern; but it may be
>> >>> that I am simply perceiving a problem that no one else believes
>> >>> exists.
>> >>>
>> >> Let me make my point again.   I'm arguing that instead of
>> >> scipy.preview,
>> >> let's just make a *single* scikit called scikit.preview or
>> >> scikit.forscipy or scikit.future_scipy or whatever.    This will create
>> >> some incentive to make scikits easier to install generally as we want
>> >> to
>> >> get the future_scipy out there being used.
>> >>
>> >> I'm very interested, though, to hear what developers of modules that
>> >> they would like to see in SciPy but have not made it there yet, think.
>> >>
>> >> I'm very interested in the question of how do we make it easier to
>> >> contribute to SciPy.
>> >
>> > As a developer who has written the module that is sparking this
>> > discussion, if the route to inclusion in scipy were "make a scikit,
>> > maintain and distribute it until you get enough user feedback to judge
>> > whether the API is optimal, then move it fully-formed into scipy" my
>> > code would simply gather dust and never be included. I don't have the
>> > time and energy to maintain a scikit.
>> That's what I don't understand: there is almost no difference between
>> maintaing a scikit and a scipy submodule. In both case you have to
>> write some setup.py + the module itself. To get the sources, it is
>> scikit vs scipy svn. Both Damian and you made this case, so I would
>> like to understand what's so different from your POV, because I just
>> don't get it ATM. Maybe there are some confusion on how a scikit can
>> be made and distributer (the documentation could certainly be
>> improved).
> I think we could make a distinction between pure python/C code without
> dependencies, and code that needs to deal with fortran compilers or library
> deficiencies. For the former, a simple setup.py using distutils should do
> the job and once that is in place it shouldn't be difficult to maintain. For
> the latter, inclusion in scipy proper might be the better route.

Good point. There is a big difference between new code that depends
heavily on external libraries, especially new ones, and
python/C-standard compliant code with no dependencies. One of the
reasons why scipy.sparse has made the release process difficult is its
dependencies. David has patiently worked through a lot of these
issues, and I thank him for it.

However, it seems we should be a bit more circumspect including code
in scipy proper that depend on external libraries. Code with few if
any dependencies seem like better candidates for inclusion. Would you
care to clarify about what you mean because it seems like you're
arguing the opposite?

One of the nice things about scipy.spatial and scipy.cluster is that
both packages require no external dependencies, putting much less
burden on the release process.

For now, it seems like deciding what to include on a case by case
basis may be the best approach. If a developer has shown a strong
presence on the mailing lists, has made general contributions to the
SciPy package, has developed well-written stable production code
tested and documented to standards, is highly likely to maintain the
package after its release, has shown that the code compiles and passes
tests on the architectures supported by SciPy, then the package is
more deserving of consideration for inclusion. Not saying these
conditions are necessary nor sufficient but they are a good rough
guidelines about what to look for in new code.


Damian Eads                             Ph.D. Student
Jack Baskin School of Engineering, UCSC        E2-489
1156 High Street                 Machine Learning Lab
Santa Cruz, CA 95064    http://www.soe.ucsc.edu/~eads

More information about the Scipy-dev mailing list