[SciPy-dev] a modest proposal for technology previews

Jarrod Millman millman@berkeley....
Tue Nov 4 03:30:04 CST 2008

On Tue, Nov 4, 2008 at 12:36 AM, David Cournapeau
<david@ar.media.kyoto-u.ac.jp> wrote:
> Jarrod Millman wrote:
>> The major difference I see is that one is a scikit and the other is
>> part of scipy.
> Yes - and that's exactly why I think it is good to use scikits for that
> :) If you put things in scipy.preview:
>    - what happens when the build breaks on common platforms ?
>    - what happens when you want to work in a manner which is not
> synchronized with scipy release process ?

Those are problems regardless of whether your code is in scipy.preview or
scipy.xxx.  I am not suggesting that we include unstable, broken code
in scipy.preview.
What I am suggesting is that for code that we agree should be in scipy
and are ready
to include in scipy, that the code first go into scipy.preview.

> The only advantage of scipy.preview I can see is that it is a easier to
> get the code for scipy users (hence my discussion about scikits build
> management). I don't think it outweights the disadvantages.

I feel like you are either misunderstanding (or perhaps dismissing) my
argument (I am fairly tired so I don't claim to be making it very
well).  The main and specific advantage is that when we are ready to
include new code into scipy, we have a staging area where we can clean
up or fix the API before just thowing it into the code base.  I don't
want to use scipy.preview as a sandbox to write new code.  If you are
starting a new package, you could use code.google.com like Damian did,
or a branch like Anne did, or make a scikit.  I don't really care how
individual authors want to do that.  But once you have some reasonably
mature code and the scipy developers agree that they would like to
include your code, then you start by including your code in

> Here is how I would see the process:
>    - you start coding your scikit
>    - once you have something, you discuss it on the ML, and you say you
> want it to include for scipy (here we can put any requirement: decent
> doc, testsuite, etc...)
>    - if the consensus is that it can be included, then put it somewhere
> in scipy
> As you see, there is almost no difference compared to putting it into
> scipy.preview from the code review process. BUT, during the dev process,
> it can break scipy build, may not buildable on the platforms we usually
> support, etc... The whole code process not happening in scipy.preview
> has a lot of advantages.

I am not suggesting that "the whole code process happen in
scipy.preview".  And there is a very concrete difference with what I
am proposing and the code review process you outline above.  Namely I
am proposing an additional step between 1) the ML discussion with
consensus and 2) put it somewhere in scipy.  That step is put it
somewhere in scipy.preview between 1) and 2).

>>  In my mind, for something to be part of scipy means
>> that it should fit into the package in a consistent way.  If I was
>> developing a scikit, I don't think I would necessarily write it the
>> same way as if I was trying to make it part of scipy.
> I don't understand this: it just means you have to think that it can go
> into scipy when you develop your scikit. How would the code would be any
> different is it was developed under scikits or scipy.preview ? The only
> difference for the source is that the namespace and the svn repository
> is different.

It wasn't a particularly important point.  I was just trying to point
out that scikit code doesn't necessarily mean that the author intends
to include the code in scipy.  It could, of course.  But again the
issue I am trying to point out is that: even if there were nightly
builds of windows, mac, linux, bsd, etc. of every scikit, the problem
I am trying to address wouldn't necessarily be addressed.
Specifically I am trying to propose a very narrowly defined way to get
a technology preview of new code to end user's as part of our regular
and official scipy releases.  I am open to discussions about what the
bar should be for getting code in as part of the technology preview.

Jarrod Millman
Computational Infrastructure for Research Labs
10 Giannini Hall, UC Berkeley
phone: 510.643.4014

More information about the Scipy-dev mailing list