[Numpy-discussion] [ANN] numscons 0.3.0 release

[David Cournapeau]

Matthew Brett wrote:
> Hi David,
>
> Thanks again for doing this.  I've been playing with the compilation -
> should I email you direct with questions, to the lists?
I don't think it would be inappropriate to use the current lists for that.
>
> Basically, I'm trying to understand the library discovery, linking
> steps - ATLAS in particular.
Don't trust the included doc: it is not up-to-date, and that's the part 
which I totally redesigned since I wrote the initial doc.

For things like FFT, BLAS, LAPACK, things are a bit complicated. 
Fortunately, for "normal" libraries, things are much easier: you use the 
NumpyCheckLibraryAndHeader function

http://projects.scipy.org/scipy/numpy/wiki/NumpySconsExtExamples#Checkingforalibrary

This function can check for a library, headers, and check for symbol in 
the library. site.cfg customization is also automatically taken care of. 
I also intend to add the possibility to check for libraries which use 
pkgconfig.

For BLAS/LAPACK/FFT, the basic principles are:
    - I make a distinction between "API libraries" (called meta 
libraries in numscons) and "implementation libraries" (called perflib in 
numscons).
    - Meta libraries uses perflibs: FFT/BLAS/LAPACK may use 
Sunperf/MKL/ATLAS.
    - For each meta library and each perflib, a Check function is 
defined: CheckMKL, CheckCBLAS, etc...
    - For a perflib check to succeed, I try to compile, link and run 
code snippets. Any failure means the Check* function will return failure.
    - When a perflib Check* succeeds, the meta library check checks that 
another code snippet can be compiled/linked/run. For example, if 
CheckMKL succeeds in CheckCBLAS, I will compile/link/run a cblas program 
using MKL configuration.
    - Both perflib and meta libraries are customizable through site.cfg 
(I tried to keep backward compatibility at this level with numpy.distutils).

I still have the feeling that it is over-designed, but BLAS/LAPACK 
libraries are really a PITA. Every perflib has slightly different 
conventions, and I wanted to use common code for all of them as much as 
possible [1]. Also, the configuration is not inside python code anymore: 
libraries, frameworks, link flags are all defined in perflib.cfg.

cheers,

David

[1] This design also allow some funky stuff: for example, when using sun 
performance libraries with sun tools, because of something which look 
like a bug to me, the sunperf are not linked when building a shared 
library. So I link a code snippet, and parse the linker output to get 
the necessary libraries; this is totally transparent to BLAS/LAPACK 
checkers.