[SciPy-dev] Package organization

Fernando Perez Fernando.Perez at colorado.edu
Thu Oct 13 02:05:58 CDT 2005

Arnd Baecker wrote:
> On Thu, 13 Oct 2005, Fernando Perez wrote:
>>This layout would allow the core team to work with relative freedom at the
>>top-level namespace, without worrying about toolkits taking names they may
>>need in the future.  Similarly, toolkit authors will have a well-defined API
>>to build upon.
> To me this looks as if there is essentially no difference
> to people distributing their projects independently of scipy
> (and outside of its name-space).
> I still like this for two reasons
> a) it would bring scientific python projects under one umbrella
> b) the (hopefully) simple user installation:

You are correct, and that's precisely the point.  Much as python's stdlib 
provides a base for functionality which all other python packages can rely on, 
scipy would do the same for scientific work.  However, I deeply dislike the 
fact that python blends all namespaces into one via $PYTHONPATH: I would much 
rather have the stlib  live under something like 'std':

from std import sys, os, time

so that what is coming from the stdlib would be explicitly separated from what 
is a third-party package (in site-packages, typically).  I'm trying to make 
such a distinction explicit, by making sure that anything that says

from scipy import foo

is the 'scipy stdlib', while

from scipy.kits import bar

explicitly indicates it's a toolkit.

In addition, since most likely all packages of serious scientific interest 
will need at the very least the core of scipy (for array facilities), having 
them under the scipy umbrella makes this explicit.

So yes, people could distribute fully outside of the scipy namespace (they 
currently do).  But by using the scipy infrastructure, they can benefit from a 
set of low-level facilities (distutils, testing, f2py, BuildSystem if 
ported,...), and they also provide a more unified interface for their users. 
Their packages can register with the top-level docstring facilities, so that 
interactive help/indexing systems show a coherent set of tools for scientific 
work, hopefully we'll have standardized conventions for code examples and 
docstrings with latex support, etc.

Could people continue to work outside of the umbrella? Absolutely.  But 
hopefully the benefits will be significant, with no major drawbacks for 
third-party package authors.

All I'm proposing is really to mimic in scipy/scipy.kits the existing 
conventions of the Python world with the stdlib/site-packages, with the minor 
addition of an explicit namespace to disambiguate (which I wish the core 
language had).

> [...]
>>If this is combined with a CPAN-like system (eggs, PyPi, whatever), it should
>>be very easy for users, once they have the basic layers in place, to grab a
>>toolkit by issuing a single command or going to a website.  I'd suggest, if
>>this were adopted, keeping a simple page at scipy with brief descriptions for
>>each toolkit, even if they are developed/distributed externally.
> Sounds very nice! I have no experience with eggs:
> where do these go when a user cannot install them as root?
> (Of course, a  PYTHONPATH modification should do the job.)

Robert is our resident expert on those little beasts, so I'll leave this one 
to him.

> [...]
>>Anyway, I've certainly far exceeded my opinion budget on this one, so I should
>>shut up now :)
> (If I disagree with this point, would this
> decrease my-already-in-the-negative-I-presume opinion budget? ;-)

And I'm headed to bankruptcy court now :)



More information about the Scipy-dev mailing list