[Numpy-discussion] Scons and numpy, second milestone: all numpy built with scons.

[David Cournapeau]

Hi,

   I am happy to announce a second milestone in scons support in numpy. 
This one is much bigger than the first one, since now I can build the 
whole numpy using scons (e.g. distutils is not used at all to build non 
python code).

How to use it:
==============

just do as before for installing numpy: python setup.py build

To use parrallel builds: python setup scons --jobs=4 build

Windows build does not work yet, but this should be easy to fix. Do not 
use it for practical purpose, though: I have not yet tackled the problem 
of optimization flags, nor the detection of blas/lapack.

What has been done:
===================

Some of the things which are now possible:
   - any module can be customized wrt build options (warning flags, 
etc...). This can be done per object file, also, with a bit some work.
   - for numpy.core (which took 95 % of my time for this milestone), all 
the generated code is integrated in the scons dependency graph. I don't 
know how distutils handled this, but if you change a txt file, or a 
header, scons will rebuilt the necessary things (the dependencies are 
found automatically). I also cleanup up some things (done in a separate 
svn branch).
   - Numpy configuration is also totally integrated into scons config 
framework, which makes the thing more understandable, I hope.

and of course, everything possible with scons is now available. Some 
other nice advantages which I did not think about at first:

   - parallel builds: since scons supports parrallel builds, I thought 
it would be interesting to support it in distutils. This was trivial to 
add, and this largely alleviates the cost added by using scons in the 
first place. For scipy, my guess would be that it could significantly 
reduce the build time (whereas numpy build time is mostly spent 
compiling one or two big files).

So basically, I believe most of the things planned in 
http://projects.scipy.org/scipy/numpy/wiki/DistutilsRevamp are now 
available because they are available in scons, if numpy developers 
decide to follow the scons route. Before being usable, I need to finish 
fortran support and blas/lapack/atlas detection: once this is done, 
numpy should be able to support some platforms not supported yet (intel 
compiler on windows, sun compiler with performance libraries, etc...), 
which was the reason I started this work in the first place.

cheers,

David