[SciPy-dev] MCMC, Kalman Filtering, AI for SciPy?

[Pearu Peterson]

On Mon, 27 Sep 2004, Charles Harris wrote:

> eric jones wrote:
>
>> Where should these live?
>> monte carlo and markov chain might fit in scipy.stats?
>
> How about in monte_carlo or some such? I think there is too much stuff 
> put in odd places. Why is zeros in optimize? Makes no sense, but there 
> it is.  I don't think now is the time to change all the directories 
> around, but I hope we give some thought to the organization before it 
> becomes unmanageable. The Dewey decimal classification was an 
> achievement I am coming to appreciate.

I agree that tools provided by Scipy are not organized well with respect 
to the mathematical subject that they deal with and so finding the needed 
tool for some specific task may not be always easy. And I agree that this 
issue should be tackled as early stage as possible in Scipy evolution, 
otherwise it will get more and more difficult to decide where to put 
contributions from the society and there is a danger of postponing such 
decisions to an unreachable future..

The current organization of Scipy packages is mostly based on underlying
Fortran/C libraries and that is obviously not the best way to organize
any high-level scientific tool.

While Chuck mentioned Dewey decimal classification then there are other
classification schemes available. For example, MSC 
(http://www.ams.org/msc/). A nice overview of MSC can be found in
   http://www.math.niu.edu/~rusin/known-math/index/index.html

I don't know how well could these classification schemes be used
for organizing Scipy packages, may be we should take a look how Matlab or 
Maple or Mathematica deal with organizing their tools.

>From the maintenance point of view, IMHO, wrappers to external Fortran/C 
libraries should be refactored from scipy packages to some "lib" 
package. For example, there would be packages like

   scipy.lib.blas
   scipy.lib.lapack
   scipy.lib.fftpack
   scipy.lib.minpack
   scipy.lib.cephes
   scipy.lib.odepack
   scipy.lib.quadpack
   scipy.lib.fitpack
   scipy.lib.minpack
   scipy.lib.superlu
   scipy.lib.amos
   scipy.lib.cdflib
   etc

and they will be used by higher level packages like

   scipy.linalg
   scipy.linalg.sparse
   scipy.stats
   scipy.optimize
   scipy.special
   etc.

These higher level packages can be organized by whatever scheme will be 
chosen, the scheme may be changed or even replaced in future, but the
core of scipy, scipy.lib, that contains most of the hard work in scipy, 
should stay as constant as possible.

Please, send suggestions on how to better organize scipy high-level 
packages.

Thanks,
Pearu