[Numpy-discussion] Bitting the bullet: using scons to build extensions inside distutils ?

David Cournapeau cournape@gmail....
Thu Sep 20 03:56:22 CDT 2007

On 9/20/07, Travis E. Oliphant <oliphant@enthought.com> wrote:
> David Cournapeau wrote:
> > Hi,
> >
> >    Starting thinking over the whole distutils thing, I was thinking
> > what people would think about using scons inside distutils to build
> > extension. The more I think about it, the more I think than distutils
> > not being maintained, and numpy/scipy building needs being much more
> > complicated (at least different) than usual python extension, trying
> > to circumvent distutils problems is an ever ending fight. Scons, being
> > developped as a Make replacement, can do all we would like to be able
> > to do with distutils, including:
> >
> Pearu is correct that numpy.distutils has quite a few features that are
> not easily replaced and so there will be some skepticism initially in
> doing things differently.

Now I understand the skepticism, my email was really not clear wrt my
intention. I do not intend to replace numpy.distutils by writing
everything from scratch: that would be stupid, it would require
several weeks of full time programming and testing on various
platforms, for no good reasons.

Keeping distutils is anyway more or less mandatory because of eggs,
which (at least now), require setuptools, which requires distutils.
Being incompatible with everybody would also be stupid.

For those reasons, I never intended to replace everything, but
gradually replacing the parts of distutils/numpy.distutils which are
limitating today (no custom library to be used with ctypes, no per
extension compilation options) by scons. If this is successfull and
people are satisfied with it, we may then gradually start to build
numpy extension themselves using scons instead of numpy.distutils
where it makes sense.

I started some work for a scons command inside numpy.distutils in
numpy.scon branch. The idea is that if you want to use scons instead
of distutils for one extension, you add the scons script in the
setup.py, and distutils then calls scons to build the extension. I
still have to figure how to get the various build directories, and how
to pass them from distutils to scons, but once this is done, there
will be a first prototype which gives an idea on how I expect to make
things work.

This alone will enable different compilation environments (under
scons, an environment is an object which keeps all the build tools
names and options, so that you can build different targets with
different environments: for example pyrex C code will have no
warnings, but by default, python extension will have full warning
enabled under gcc, etc...) and ctypes support in a cross platform way
(to build our own C libraries callable from ctypes).


More information about the Numpy-discussion mailing list