[SciPy-dev] Package organization

[Travis Oliphant]

I really like the discussion that is occurring.  Part of the problem 
with the current system is that there was no discussion and so it just 
evolved.   I'm not against major surgery on the organization as long as 
there is some mechanism for helping old users move forward.   That's why 
scipy has not reached 1.0 yet -- because of code organization issues.

First.   I like what Fernando is trying to do with the scipy.kits 
concept.   The most valuable thing scipy could provide is a mechanism 
for auto-indexing your scipy-related packages for a help browser.  I 
don't think packages would need to live under scipy to accomplish this 
(but it would make it easier).   There are other mechanism like the 
environment variable concept that could also collect package 
information.  In fact, I wonder if scipy could detect that some module 
imported it and add that to some internal index (or only add it if the 
module defined some __scipy__  name or something...



Second.  Regarding the debate over the .lib directory.   I could accept 
the fact that an extremely flat packaging structure has advantages.   
After all MATLAB "gets away" with an enormously flat system.  I also see 
the dependency problem with a hierarchial approach.  I think Robert 
raises a valid point, that a nested hierarchy is largely just a 
documentation/indexing issue.  It  could be handled using some standard 
form of keywords/GAMS classification numbers in the sub-packages 
themselves.

For example in optimize.py

__gamskeys__ = {'fmin': 'G1a1a'}

Then scipy could provide an indexing system that would pick all of these 
names up and produce a GAMS hierarchy of code available.

I think one of the purpose of the .lib directory is to extract those 
elements of the current full scipy that cause interdependencies among 
the modules to a 'library' directory that holds the actual dependencies. 
  It could still serve that purpose.    If we just start moving things 
to the library on a case-by-case basis, it might help us decide what 
should really go there. 

While I don't really have enough data to form a strong opinion, I like 
the idea of a flat structure under scipy with perhaps a few 
sub-directories like lib and sandbox.

Third:  I have no problem with moving ode solvers out of integrate into 
another sub-package.


-Travis