[SciPy-dev] scipy distutils questions

[Arnd Baecker]

Hi,

yesterday I ran into a bunch of problems related to flapack/clapack
which I tried to work-around via distutils. Some of those attempts
did not work, so I would like to ask if I did something wrong here.
Apologies ahead if this is already mentioned in some
documtation - a pointer would be very much appreciated in that
case.

a) The Intel Math Kernel library provides LAPACK in both
   double (libmkl_lapack64.so) and single (libmkl_lapack32.so)
   precision.
   With the following in `site.cfg`

     [atlas]
     library_dirs = /opt/intel/mkl72/lib/64
     include_dirs = /opt/intel/mkl72/include/
     atlas_libs = mkl_lapack64,libmkl_lapack32,mkl

     [lapack]
     library_dirs = /opt/intel/mkl72/lib/64
     include_dirs = /opt/intel/mkl72/include/
     lapack_libs = mkl_lapack64,libmkl_lapack32

   it was not able to add them (without `libmkl_lapack32` the double
   precision one worked).
   Would that in principle be the correct way?

b) As this did not work, I changed in `Lib/linalg/setup.py`
  `skip_single_routines = True`
  This only had an effect (in the sense of rebuilding
  lapack/flapack/...) when  I touched the corresponding pyf files,
      touch *.pyf
  ((Is this mentioned somewhere in the docs?
    If so, I completely missed that))
   Is it also correct that
   removing the contents of the `linalg` directory in the `build`
   directory does not force a rebuild?

c) It seems that not all s* routines are listed in linalg/setup.py.
   I had to add the following in Lib/linalg/setup.py:
   skip_names['clapack'].extend(\
            'sgesv sgetrf sgetrs  cgesv cgetrf   cgetrs
             ...
   skip_names['flapack'].extend(\
            'sgesv sgebal sgbsv sgehrd  cgbsv  cgebal   cgehrd
             ...

   Would it possible to extract/exclude those automatically?

d) running the usual `python setup.py  .... install  --prefix=...`
   rebuilds the corresponding libraries (e.g _flapack.so).
   ((I always removed them before, but that might not be necessary))

   However, the resulting libararies were not moved to
   the destination, given by prefix.
   I had to copy them manually.
   Is this to be expected?

If all the above behaviour is really unavoidable
(eg. by more clever command line options etc.),
then I think the following would help a lot:
- ad: b) if setup.py is newer than the files which are generated,
  they should be rebuild.
- ad: c) it would be good, if changed files get copied
  over to the destination.

In addition: could you pleas add `sample_site.cfg` again
(or/and site.cfg, with everything commented out)

Because of all this (and previous problems concerning exchanging
fft libraries, presumably for similar reasons)
I presently completely remove the destination tree and the build
tree (sometimes I even do a full new checkout, if everything seems
screwed).

One more remark/question:
Usually scipy (core) will not require to add a site.cfg.
However for full scipy adding a site.cfg is often needed.
It seems that this file has to be placed into the
  site-packages/scipy/distutils/
directory or are there other places?
(For example couldn't it look in the directory from which
the `python setup.py install` command is issued?)

Best, Arnd