[Numpy-discussion] Build questions, atlas, lapack...

Fernando Perez fperez.net at gmail.com
Thu Jun 8 17:48:27 CDT 2006


Hi all,

I'm starting the transition of a large code from Numeric to numpy, so
I am now doing a fresh build with a lot more care than before,
actually reading all the intermediate messages.  I am a bit puzzled
and could use some help.

This is all on an ubuntu dapper box with the atlas-sse2 packages (and
everything else recommended installed).  By running as suggested in
the scipy readme:

python ~/tmp/local/lib/python2.4/site-packages/numpy/distutils/system_info.py


I get the following message at some point:


====================================
atlas_info:
( library_dirs = /usr/local/lib:/usr/lib )
( paths: /usr/lib/atlas,/usr/lib/sse2 )
  looking libraries f77blas,cblas,atlas in /usr/local/lib but found None
  looking libraries f77blas,cblas,atlas in /usr/local/lib but found None
  looking libraries lapack_atlas in /usr/local/lib but found None
  looking libraries lapack_atlas in /usr/local/lib but found None
  looking libraries f77blas,cblas,atlas in /usr/lib/atlas but found None
  looking libraries f77blas,cblas,atlas in /usr/lib/atlas but found None
  looking libraries lapack_atlas in /usr/lib/atlas but found None
  looking libraries lapack_atlas in /usr/lib/atlas but found None
( paths: /usr/lib/sse2/libf77blas.so )
( paths: /usr/lib/sse2/libcblas.so )
( paths: /usr/lib/sse2/libatlas.so )
( paths: /usr/lib/sse2/liblapack_atlas.so )
  looking libraries lapack in /usr/lib/sse2 but found None
  looking libraries lapack in /usr/lib/sse2 but found None
  looking libraries f77blas,cblas,atlas in /usr/lib but found None
  looking libraries f77blas,cblas,atlas in /usr/lib but found None
  looking libraries lapack_atlas in /usr/lib but found None
  looking libraries lapack_atlas in /usr/lib but found None
system_info.atlas_info
( include_dirs = /usr/local/include:/usr/include )
( paths: /usr/include/atlas_misc.h,/usr/include/atlas_enum.h,/usr/include/atlas_aux.h,/usr/include/atlas_type.h
)
/usr/local/installers/src/scipy/numpy/numpy/distutils/system_info.py:870:
UserWarning:
*********************************************************************
    Could not find lapack library within the ATLAS installation.
*********************************************************************

  warnings.warn(message)
( library_dirs = /usr/local/lib:/usr/lib )
( paths: /usr/lib/atlas,/usr/lib/sse2 )
  FOUND:
    libraries = ['f77blas', 'cblas', 'atlas']
    library_dirs = ['/usr/lib/sse2']
    language = c
    define_macros = [('ATLAS_WITHOUT_LAPACK', None)]

====================================

What I find very puzzling here is that  later on, the following goes by:

lapack_atlas_info:
( library_dirs = /usr/local/lib:/usr/lib )
( paths: /usr/lib/atlas,/usr/lib/sse2 )
  looking libraries lapack_atlas,f77blas,cblas,atlas in /usr/local/lib
but found None
  looking libraries lapack_atlas,f77blas,cblas,atlas in /usr/local/lib
but found None
  looking libraries lapack_atlas in /usr/local/lib but found None
  looking libraries lapack_atlas in /usr/local/lib but found None
  looking libraries lapack_atlas,f77blas,cblas,atlas in /usr/lib/atlas
but found None
  looking libraries lapack_atlas,f77blas,cblas,atlas in /usr/lib/atlas
but found None
  looking libraries lapack_atlas in /usr/lib/atlas but found None
  looking libraries lapack_atlas in /usr/lib/atlas but found None
( paths: /usr/lib/sse2/liblapack_atlas.so )
( paths: /usr/lib/sse2/libf77blas.so )
( paths: /usr/lib/sse2/libcblas.so )
( paths: /usr/lib/sse2/libatlas.so )
( paths: /usr/lib/sse2/liblapack_atlas.so )
  looking libraries lapack in /usr/lib/sse2 but found None
  looking libraries lapack in /usr/lib/sse2 but found None
  looking libraries lapack_atlas,f77blas,cblas,atlas in /usr/lib but found None
  looking libraries lapack_atlas,f77blas,cblas,atlas in /usr/lib but found None
  looking libraries lapack_atlas in /usr/lib but found None
  looking libraries lapack_atlas in /usr/lib but found None
system_info.lapack_atlas_info
( include_dirs = /usr/local/include:/usr/include )
( paths: /usr/include/atlas_misc.h,/usr/include/atlas_enum.h,/usr/include/atlas_aux.h,/usr/include/atlas_type.h
)
( library_dirs = /usr/local/lib:/usr/lib )
( paths: /usr/lib/atlas,/usr/lib/sse2 )
  FOUND:
    libraries = ['lapack_atlas', 'f77blas', 'cblas', 'atlas']
    library_dirs = ['/usr/lib/sse2']
    language = c
    define_macros = [('ATLAS_WITH_LAPACK_ATLAS', None)]

==============================================

Does the second mean that it /is/ finding the right libraries?  Since
the first search in atlas_info is also printing

( paths: /usr/lib/sse2/liblapack_atlas.so )

I don't quite understand why it then reports the warning.

For reference, here's the content of the relevant directories on my system:

==============================================
longs[sse2]> ls /usr/lib/sse2
libatlas.a       libcblas.a       libf77blas.a       liblapack_atlas.a
libatlas.so@     libcblas.so@     libf77blas.so@     liblapack_atlas.so@
libatlas.so.3@   libcblas.so.3@   libf77blas.so.3@   liblapack_atlas.so.3@
libatlas.so.3.0  libcblas.so.3.0  libf77blas.so.3.0  liblapack_atlas.so.3.0


longs[sse2]> ls /usr/lib/atlas/sse2/
libblas.a    libblas.so.3@   liblapack.a    liblapack.so.3@
libblas.so@  libblas.so.3.0  liblapack.so@  liblapack.so.3.0

==============================================


In summary, I don't really know if this is actually finding what it
wants or not, given the two messages.

Cheers,

f

ps - it's worth mentioning that the sequence:

python ~/tmp/local/lib/python2.4/site-packages/numpy/distutils/system_info.py

gets itself into a nasty recursion where it fires the interactive
session 3 times in a row.  And in doing so, it splits its own output
in a funny way:

[...]

blas_opt_info:
========================================================================
Starting interactive session
------------------------------------------------------------------------

Tasks:
  i       - Show python/platform/machine information
  ie      - Show environment information
  c       - Show C compilers information
  c<name> - Set C compiler (current:None)
  f       - Show Fortran compilers information
  f<name> - Set Fortran compiler (current:None)
  e       - Edit proposed sys.argv[1:].

Task aliases:
  0         - Configure
  1         - Build
  2         - Install
  2<prefix> - Install with prefix.
  3         - Inplace build
  4         - Source distribution
  5         - Binary distribution

Proposed sys.argv =
['/home/fperez/tmp/local/lib/python2.4/site-packages/numpy/distutils/system_info.py']

Choose a task (^D to quit, Enter to continue with setup):

##### msg:

( library_dirs = /usr/local/lib:/usr/lib )
  FOUND:
    libraries = ['f77blas', 'cblas', 'atlas']
    library_dirs = ['/usr/lib/sse2']
    language = c
    define_macros = [('NO_ATLAS_INFO', 2)]

=================

I tried to fix it, but the call sequence in that code is convoluted
enough that after a few 'import traceback;traceback.print_stack()'
tries I sort of gave up.  That code is rather (how can I say this
nicely) pasta-like :), and thoroughly uncommented, so I'm afraid I
won't be able to contribute a cleanup here.

I think this tool should run by default in a mode with NO attempt to
fire a command-line subsystem of its own, so users can simply run

python /path/to/system_info > system_info.log

for further analysis.




More information about the Numpy-discussion mailing list