[SciPy-dev] Re: [Numpy-discussion] Numeric3
oliphant at ee.byu.edu
Fri Feb 4 19:36:24 CST 2005
> I have in the past tried to install SciPy on OSX and I had big
> problems with it. Motivated by the recent discussions on the list I
> tried again today, and found it much improved, but still quite
> involved. It would still be a pain to distribute software that has a
> dependency on SciPy to 'average users' that would have trouble
> installing SciPy. There does not seem to be a binary distribution on
> scipy.org for OS X, which would solve that problem. This is not
> unimportant, since I guess OS X is gaining popularity in scientific
A binary for OS X is a desireable thing. I think more of a stink should
be made, so that one gets provided by default. I know that Robert Kern
has worked on such a thing and it is not out of the question, but is on
the horizon. More users asking for it would help.
>> I'm very glad to hear these comments, as I constantly wonder why
>> people seem to be duplicating SciPy's efforts instead of helping SciPy.
> I agree with that, but fact is that the split between Numarray and
> Numeric prevented me from contributing to SciPy. I have developed for
> numarray (the image processing package nd_image, part of numarray, is
> done by me) because I felt numarray was much better then Numeric for
> that application, and that also ruled out SciPy. There seems now to be
> an effort going to either try to merge the two or to bring Numeric up
> to numarray standard. Thats great, and if that works out I will try to
> rewrite my package so it works with both. Would be happy to
> contribute it to SciPy. But I would make sure it remains a package
> that can be installed separately from SciPy if required.
"What scipy is" is quite open to interpretation. The idea of scipy is
just to have a super-package where different packages can live and have
some kind of inter-operability. It is made so that sub-packages can be
easily added and built upon request. We try hard to make sure that the
only real dependency is scipy_base which should not be hard to install
anywhere (does not require ATLAS or FORTRAN).
It would be great to add nd_image as a sub-package to SciPy. In fact,
nd_image was one of the things that most alarmed me about the emerging
split in the community, as it reproduces some of what is already in
SciPy (N-D convolution) while adding some great things that I've always
wanted in SciPy (morphology).
I believe scipy can be easily installed as a collection of packages if
that is desired. Pearu has done an incredible job with scipy_disutils
to make that process simple. Making a single super package helps users
see that they have only a part of what's available (but maybe all they
need), and you get the added benefit of standard documentation
efforts. Nobody has taken up the ball to actually try and distribute
individual packages of scipy, but it is a welcome effort.
www.scipy.org has places where such packages could live. Enthought has
mainly been interested in distributing "batteries-included" packages for
windows because it is what their customers need. Other are welcome to
pick up the slack.
Perhaps it would be good to get a list of which platforms actually need
binaries for people to look more favorably at SciPy. I can see that OS
X is on the top of that list, surely somebody could distribute a copy of
a binary for that platform (I know of people who have it installed).
Also, scipy is still version 0.3 not because the code is unstable but
because the layout of the packages is still open to change. Come and
discuss your visions of what it should look like with us on the
scipy-dev at scipy.net mailing list.
> Making SciPy modular is in my opinion a very good approach. Why do I
> need to install all of it, if I really only need a few parts? If I
> can install only the subset I need, the likelihood that I will have an
> easier job doing that will increase. That would make it also possible
> to have sub-sets that are free of dependencies such as FORTRAN.
> On the other hand, I would think it would be a good idea to write
> software that does not have dependencies on big packages such as
> SciPy when possible, but only on Numeric/Numarray. That does not
> exclude them from being included in SciPy. It seems to me that it
> would be a desirable property if a package can be installed just with
> Numeric (or numarray), with an appropriate sub-set of SciPy, or come
> with the big SciPy package. I would rather see a "collection of SciPy
> packages" then a "SciPy package".
I'd like to see more of the development work that goes on happen under a
common standard, that is all. SciPy has done a lot of work to provide
an infrastructure within which to distribute packages. The field is
built, but people don't seem to want to come. I understand that the
Numeric / Numarray dichotomy is a big source of this problem. That is
the whole reason I'm devoting a big portion of my time to the problem
My very ambitious goal with Numeric3 is to replace both Numeric and
Numarray (heavily borrowing code from each). When I say replace them,
I'm talking about the array object and umath module. I'm not talking
about the other packages that use them. I want those to live under a
common standard (i.e. scipy). It would be fine with me if that common
standard include a scipy_lite or something like that which was easy to
install, if that is desired. But, if the problem is really just
availability of binary copies, then why don't we solve that instead of
worrying about installation on platforms that nobody uses. In
addition, even though I've developed a lot of scipy, I'm willing to
throw it all out or alter it considerably in order to satisfy the
demands that exist for package developers to want to put their weight
behind something like SciPy. So, don't equate history with an
unwillingness to cooperate.
I think you'll find the people involved with scipy to be incredibly
willing to cooperate to achieve mutual success. If you have users of a
package that demand a certain binary of scipy to be installed, then
bring them over. I can almost guarantee you that a copy of scipy will
be available for them.
While I do recognize the value of repeated work. When it comes to
interoperability, single standards seem to work best. So, the longer
this unnecessary division of resources goes on, the farther behind
Python will get for doing anything but niche work.
More information about the Scipy-dev