[Numpy-discussion] For review: first milestone of scons support in numpy
Thu Oct 11 04:45:18 CDT 2007
David Cournapeau wrote:
> Pearu Peterson wrote:
>> 2) It seems that scons does not interfare with numpy.distutils much.
>> If this is true and numpy/scipy builds will not break when scons is
>> not installed then I think you could continue the scons support
>> development in trunk.
> It won't break if scons is not installed because scons sources are
> copied into the branch. Scons developers explicitely support this:
> (AFAIK, it does not pose any problem license-wise, since scons is new
> BSD license; it adds ~350 kb of compressed source code to numpy).
I think this is good.
Does scons require python-dev? If not then this will solve one of the
frequent issues that new users may experience: not installed distutils.
> Now, concerning migrating all compiled extensions to scons, I would
> prefer avoiding doing it in the trunk; but I heard horror stories about
> subversion and merging, so maybe staying outside the trunk is too risky
I think numpy is quite stable now that it's safe to develop in a branch
(if trunk is very actively developed then merging branches
can be a nightmare). However, IMHO using a branch makes other
developers to stay aside from branch development and in time it is
more and more difficult to merge.
> Also, when I said that I don't see big problems to replace distutils
> for compiled extensions, I lied by over-simplification. If scons is used
> for compiled extension, with the current design, distutils will call
> scons for each package. Calling scons is expensive (importing many
> modules, etc...: this easily takes around one second for each non
> trivial Sconscript), and also, because each of them is independent, we
> may check several times for the same thing (fortran compiler, etc...),
> which would really add up to the build time.
> I see two approaches here:
> - do not care about it because numpy is unlikely to become really bigger
> - considering that numpy is really one big package (contrary to
> scipy which, at least in my understanding, is gearing toward less
> inter-package dependencies ?), we should only have one big sconscript
> for configuration (checking blas/lapack, compilers, etc...), and use
> scons recursively. In this case, it should not be much slower than the
> current system.
Note that building a subpackage in subpackage directory must be
supported. So a big sconscript may not be an option.
The third approach would be to cache those checks that are called
frequently to, say, $HOME/.numpy/scons.
numpy.distutils setup.py script gathers the information from
subpackage setup.py scripts recursively and then passes all the
information to one setup function call.
I think setupscons.py should do the same. If scons does not support
recursive reading of scons scripts then the corresponding feature
should be implemented, I guess it would not be difficult.
>> 4) though, we cannot remove numpy.distutils for backward compatibility
>> with software using numpy.distutils. However, it should be possible
>> to enhance Configuration class to generate the corresponding SConstruct
>> scripts. It will save some work when converting configuration()
>> functions to SConstruct scripts.
> Could you elaborate on this point ? I am not sure what you mean by
> generating Configuration class ?
I meant that Configuration class could have a method, say
toscons(<filename>), that will generate SConstruct script
from the information that Configuration instance holds. I thought that
this would just ease creating SConstruct scripts from
existing setup.py files.
More information about the Numpy-discussion