[SciPy-dev] Package organization
Fernando.Perez at colorado.edu
Thu Oct 13 12:59:48 CDT 2005
Robert Kern wrote:
> I'm not entirely convinced that everything needs to live in the scipy
> namespace, though. I think I overstated the build-time benefits of being
> in scipy itself. I think we can make most of those benefits accessible
> to packages that aren't living in scipy.* . I haven't been keeping up
> with the AstroPy discussions, but I doubt they'd really want to do
> from scipy.kits.astropy import wcs
> etc. "Flat is better than nested," and all that.
Yes, but there's a dark side to that motto, rarely mentioned: 'too flat leads
to name clashes and silent shadowings'. That's exactly what happens with
python: the search space is a single, flat list searched in order (as
specified in some internal build-time paths extended with $PYTHONPATH,
resulting in sys.path). This means that at any time, the appearance of a new
package with the same name as another in a directory which is listed earlier
in sys.path can shadow the older package with no warning.
I think that scipy is a large enough collection of tools that it's worth
creating at least one level of namespace protection for us against the above.
Indeed, it is shorter to type
from astropy import wcs
and perhaps the astropy community (and others) would find the scipy.kits
prefix annoying, I don't know. But I do see merit to having a namespace
umbrella that may justify that price to be paid:
- scipy.kits? or help(scipy.kits) will show you all the toolkits you have
installed. I think this can help new users a lot, and even experienced ones,
as scipy grows. Think of how much people love the integrated help indices in
Mathematica or matlab...
- automatic documentation facilities can collect and generate html/pdf docs of
the scipy core/full AND the existing toolkits, including code examples.
- with the addition of a single environment variable (say SCIPY_KIT_PATH), we
can even allow users to manage local collections of toolkits outside of the
sysadmin-controlled filesystem (critical for non-root usage). Obviously
PYTHONPATH already provides this for import, but explicitly designating a
toolkit search path will tell scipy where to look at runtime for toolkits, for
the purposes of documentation/help/indexing (much like MayaVi allows for
user-level local extensions for data sources, filters and modules).
If the overall idea seems to anyone to have merit but the issue is with too
much nesting, would a simple renaming help?
from scikit.astro import wcs
for example. It does bring the nesting cost down to a single lookup level
(one dot instead of two, which means one less dictionary lookup) and 7
> I *think* a package can get all the benefits of "being in scipy" simply
> by depending on parts of scipy and using the tools appropriately. But to
> really figure this out, we need to come down to cases. Let's get our
> house in order first, and then we can talk about how we deal with guests.
I just want to make sure that we at least build a guest room in the house,
instead of asking the guests to sleep in the garage :) We can paint it later,
but we should at least plan for its existence now.
Anyway, if people don't like the idea I'll just drop the discussion so that
those coding can get back to their job, and I can get back to the
long-abandoned ipython. I'll live with the occasional PYTHONPATH-induced name
clash, but I do think it would be worth taking this opportunity now to
future-proof scipy a little in this regard.
More information about the Scipy-dev