[SciPy-Dev] Proposal for Scikit-Signal - a SciPy toolbox for signal processing

Robert Kern robert.kern@gmail....
Tue Jan 3 05:47:55 CST 2012

On Tue, Jan 3, 2012 at 09:00, Travis Oliphant <travis@continuum.io> wrote:
> I don't know if this has already been discussed or not.   But, I really don't understand the reasoning behind "yet-another-project" for signal processing.   That is the whole-point of the signal sub-project under the scipy namespace.   Why not just develop there?  Github access is easy to grant.
> I must admit, I've never been a fan of the scikits namespace.  I would prefer that we just stick with the scipy namespace and work on making scipy more modular and easy to distribute as separate modules in the first place.   If you don't want to do that, then just pick a top-level name and use it.
> I disagree with Gael that there should be a scikits-signal package.   There are too many scikits already that should just be scipy projects (with scipy available in modular form).    In my mind, almost every scikits- project should just be a scipy- project.   There really was no need for the scikits namespace in the first place.

To be fair, the idea of the scikits namespace formed when the
landscape was quite different and may no longer be especially
relevant, but it had its reasons. Some projects can't go into the
monolithic scipy-as-it-is for license, build, or development cycle
reasons. Saying that scipy shouldn't be monolithic then is quite
reasonable by itself, but no one has stepped up to do the work (I took
a stab at it once). It isn't a reasonable response to someone who
wants to contribute something. Enthusiasm isn't a fungible quantity.
Someone who just wants to contribute his wrapper for whatever and is
told to first go refactor a mature package with a lot of users is
going to walk away. As they should.

Instead, we tried to make it easier for people to contribute their
code to the Python world. At the time, project hosting was limited, so
Enthought's offer of sharing scipy's SVN/Trac/mailing list
infrastructure was useful. Now, not so much.

At the time, namespace packages seemed like a reasonable technology.
Experience both inside and outside scikits has convinced most of us
otherwise. One thing that does not seemed to have changed is that some
people still want some kind of branding to demonstrate that their
package belongs to this community. We used the name "scikits" instead
of "scipy" because we anticipated confusion about what was in
scipy-the-monolithic-package and what was available in separate
packages (and since we were using namespace packages, technical issues
with namespace packages and the non-empty scipy/__init__.py file).

You don't say what you think "being a scipy- project" means, so it's
hard to see what you are proposing as an alternative.

> Signal processing was the main thing I started writing SciPy for in the first place.   These are the tools that made Matlab famous and I've always wanted Python to have the best-of-breed algorithms for.     To me SciPy as a project has failed if general signal processing tools are being written in other high-level packages.   I've watched this trend away from common development in SciPy in image processing, machine learning, optimization, and differential equation solution with some sadness over the past several years.    Frankly, it makes me want to just pull out all of the individual packages I wrote that originally got pulled together into SciPy into separate projects and develop them individually from there.   Leaving it to packaging and distribution issues to pull them together again.
> Hmm.. perhaps that is not such a bad idea.   What do others think?  What should really be in core SciPy and what should be in other packages?   Perhaps it doesn't matter now and SciPy should just be maintained as it is with new features added in other packages?   A lot has changed in the landscape since Pearu, Eric, and I released SciPy.    Many people have contributed to the individual packages --- but the vision has waned for the project has a whole.     The SciPy community is vibrant and alive, but the SciPy project does not seem to have a coherent goal.   I'd like to see that changed this year if possible.
> In working on SciPy for .NET, I did a code.google search for open source packages that were relying on scipy imports.   What I found was that almost all cases of scipy were:  linalg, optimize, stats, special.   It makes the case that scipy as a packages should be limited to that core set of tools (and their dependencies).   All the other modules should just be distributed as separate projects / packages.

As you say, the landscape has changed significantly. Monolithic
packages are becoming less workable as the number of things we want to
build/wrap is increasing. Building multiple packages that you want has
also become marginally easier. At least, easier than trying to build a
single package that wraps everything you don't want. It was a lot
easier to envision everything under the sun being in scipy proper back
in 2000.

I think it would be reasonable to remake scipy as a slimmed-down core
package (with deprecated compatibility stubs for a while) with a
constellation of top-level packages around it. We could open up the
github.com/scipy organization to those other projects who want that
kind of branding, though that still does invite the potential
confusion that we tried to avoid with the "scikits" name. That said,
since we don't need to fit it into a valid namespace package name,
just using the branding of calling them a "scipy toolkit" or "scipy
addon" would be fine. Breaking up scipy might help the individual
packages develop and release at their own pace.

But mostly, I would like to encourage the idea that one should not be
sad or frustrated when people contribute open source code to our
community just because it's not in scipy or any particular package (or
for that matter using the "right" DVCS). The important thing is that
it is available to the Python community and that it works with the
other tools that we have (i.e. talks with numpy). If your emotional
response is anything but gratitude, then it's unworthy of you.

Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco

More information about the SciPy-Dev mailing list