[SciPy-dev] Package organization
perry at stsci.edu
Mon Oct 17 15:20:12 CDT 2005
I've finally had a chance to read over the messages on this topic and
thought I would make a few comments. It's certainly possible that I
misunderstand some of the points already made so I'll summarize some
(not all) of the issues raised, at least those I have an interest in,
as well as asking for clarification for those issues that I didn't see
clearly spelled out.
1) flatter is better than deeper. I'm inclined to agree with this. The
objection is that flat will lead to many name collisions within scipy.
I guess this depends on the next issue.
2) are all the items within the scipy namespace controlled? Some of the
discussions suggested that mechanisms be provided to extend the search
path for scipy libraries and such. I guess the question I have is why
such libraries should be loaded into the scipy namespace. Nothing
prevents them from having dependencies on scipy, but do they actually
need to be loaded into its namespace? I can't think of any reason why.
If not, then that means the scipy project is the master of the
namespace and should detect when someone tries to add a duplicate name.
3) This doesn't mean that naming problems won't arise. Using the
example given regarding astropy, I'm not sure that an astropy wcs
package should just go right into the scipy namespace. That acronym
might be used by other specialties in entirely different ways. In this
case there is something to be said for qualifying the name somehow.
That might be with the name itself, i.e., astrowcs. or that domain
specific knowledge does justify some nesting. Perhaps we should
consider a policy where mathematical algorithms are treated as general.
After all, they may be used in many specialties of science and
engineering. So it is important not to group them by kind of algorithm,
or by those that might use it. On the other hand, there are many
domain-specific areas where it is unlikely that the package or module
will be used in another. I doubt that many other areas (but in fact, a
few do) will use the FITS format. A library like that could reasonably
be confined to its own namespace (like astrolib). I don't know if there
is any simple approach to this. It seems to me some judgment is
required. One could argue that something like astronomical-specific
libraries shouldn't fall under scipy at all but in its own package.
4) So why not start flat with algorithms, and leave domains to define
their own name space as a starting point? If it is necessary to move
things into scipy because they have usefulness over more than one
domain, then do that when the need arises. Being conservative on this
can probably put off name collision issue for a good while.
4a) But as pointed out, there are occasions where one wants a generic
module or package (e.g., fft) for which there may be many
implementations. With the flat approach, the generic name should be
reserved for some sort of redirection mechanism to use a more specific
5) One of the points raised is that lib can be used to indicate items
for which many other packages have dependencies. Perhaps. I suspect
that things won't be so clear. Maybe the right thing to do is just to
provide tools to give packages an easy way of ensuring that all needed
dependent scipy elements are already installed and informing the user
which are missing (ideally, automatically installing them as needed,
but I don't know if we need to be that ambitious at the start). Perhaps
a sandbox is useful to the extent it doesn't force a final name choice.
6) Then again, I could be entirely confused about the real issues :-)
More information about the Scipy-dev