[SciPy-dev] Re: [Numpy-discussion] Trying out Numeric3
pearu at scipy.org
Thu Mar 24 23:58:43 CST 2005
On Thu, 24 Mar 2005, Travis Oliphant wrote:
> Michiel Jan Laurens de Hoon wrote:
>> Arnd's comment raises the question of how to try out or contribute to
>> Numeric3 if the code base is changing from day to day. It may be a good
>> idea to set up some division of labor, so we can contribute to Numeric3
>> without getting in each other's way. For example, I'd be interested in
>> working on setup.py and putting different parts of Numeric3/scipy_base
> Michiel, you are free to work on setup.py all you want :-)
> Putting the parts of scipy_base together is a good idea. Exactly how to
> structure this is going to require some thought and need to be coordinated
> with current scipy.
> I want a package that is as easy to install as current Numeric (so the
> default will have something like lapack_lite).
> But, this should not handicap nor ignore a speed-conscious user who wants to
> install ATLAS or take advantage of vendor-supplied libraries.
> There should be a way to replace functionality that is clean and does not
> require editing setup.py files.
> Anybody with good ideas about how to do this well is welcome to speak up.
> Perhaps, the easiest thing to do is to keep the basic Numeric structure (with
> C-based easy-to-install additions) and call it scipylite (with backwards
> compatibility provided for Numeric, LinearAlgebra, RandomArray, and MLab
> names). This also installs the namespace scipy which has a little
> intelligence in it to determine if you have altas and fortran capabilities
> installed or not.
> Then, provide a scipyatlas package that can be installed to take advantage of
> atlas and vendor-supplied lapack/blas.
> Then, a scipyfortran package that can be installed if you have a fortran
> compiler which provides the functionality provided by fortran libraries.
> So, there are three divisions here.
Hmm, the idea of introducing scipylite, scipyatlas, scipyfortran packages
does not sound like a good idea. The usage of atlas or fortran blas/lapack or
vendor based blas/lapack libraries is an implementation detail and should
not be reflected in scipy_base package structure. This is because such an
approach is not suitable for writing portable Numeric3 based applications
or packages. For example, if a developer uses scipyfortran package in a
package, it immidiately reduces the number of potential users for this
I got an impression from earlier threads that scipy_distutils will be
included to scipy_base. So, I am proposing to use scipy_distutils tools
and our scipy experience for dealing with this issue, scipy.lib.lapack
would be a good working prototype here.
Ideally, scipy_base should provide a complete interface to LAPACK
routines, but not immidiately, of course. Now, depending on the
availability of compilers and resources in a particular computer, the
following would happen:
1) No Fortran compiler, no lapack libraries in the system, only C compiler
is available --- f2c generated lite-lapack C sources are used to build
lapack extension module; wrappers to lapack routines, for which there are
no f2c generated sources, are disabled by f2py `only:` feature.
lite-lapack C sources come with scipy_base sources.
2) No Fortran compiler, system has lapack libraries (atlas or Accelerate
or vecLib), C compiler is available --- system lapack library will be used
and a complete lapack extension module can be built.
3) Fortran and C compiler are available, no lapack libraries in the system
--- Fortran lite-lapack sources are used to build lapack extension module;
lite-lapack Fortran sources come with scipy_base sources. Similar to
the case (1), some wrappers are disabled.
4-..) other combinations are possible and users can choose their favorite
The availability of system resources can be checked using
scipy_distutils.system_info.get_info. Checking the availability of Fortran
compiler should be done in a configuration step and only when an user
specifically asks for it, by default we should assume that Fortran
compiler is not available. The same should apply also to atlas/lapack/blas
libraries, by default f2c generated lite-lapack C sources will be used.
In this way users that only need Numeric3 array capabilities will avoid
all possible troubles that may show up when using all possible resources
for speed on an arbitrary computer.
Btw, I would suggest using `scipy <dot> <package>` instead of
`scipy <package>` or `scipy <underscore> <package>` for naming packages.
More information about the Numpy-discussion