[Numpy-discussion] Unable to building numpy with openblas using bento or distutils

Skipper Seabold jsseabold@gmail....
Sat Mar 23 13:19:42 CDT 2013


Some help on this would be greatly appreciated. It's been recommended to
use OpenBlas over ATLAS, so I've been trying to build numpy with openblas
and have run into a few problems.

1) Build fails using bento master and waf 1.7.9, see below.
2) Distutils doesn't seem to be able to find lapack as part of atlas. I
tried to skip a site.cfg and define environmental variables. No idea what I
missed.

I followed instructions found scattered over the internet and only
understand vaguely the issues. Maybe someone can help. I'd be happy to
update the wiki with any answers.

To truly support OpenBlas, is it maybe necessary to make some additions to
numpy/distutils/system_info.py?

Thanks for having a look,

Skipper

Install OpenBlas
-----------------------------
git clone git://github.com/xianyi/OpenBLAS
cd OpenBlas

Edit c_check to look for libpthreads in the right place (Kubuntu 12.10)

|4 $ git diff c_check
```
diff --git a/c_check b/c_check
index 4d82237..de0fd33 100644
--- a/c_check
+++ b/c_check
@@ -241,7 +241,7 @@ print CONFFILE "#define FUNDERSCORE\t$need_fu\n" if
$need_fu

 if ($os eq "LINUX") {

-    @pthread = split(/\s+/, `nm /lib/libpthread.so* | grep
_pthread_create`);
+    @pthread = split(/\s+/, `nm /lib/x86_64-linux-gnu/libpthread.so* |
grep _pthread_create`);

     if ($pthread[2] ne "") {
        print CONFFILE "#define PTHREAD_CREATE_FUNC     $pthread[2]\n";
```

make fc=gfortran
make PREFIX=~/.local install

Everything looks ok, so far.

Install NumPy
---------------------------
Using numpy master

I tried to use bento master and waf 1.7.9, following instructions from
David's blog

bentomaker configure --prefix=/home/skipper/.local
--with-blas-lapack-libdir=/home/skipper/.local/lib
--blas-lapack-type=openblas ..
bentomaker build -j4

```
<snip>
[101/104] cshlib: build/numpy/core/src/umath/umath_tests.c.11.o ->
build/numpy/core/umath_tests.so

/usr/bin/ld: numpy/core/lib/libnpymath.a(halffloat.c.12.o): relocation
R_X86_64_PC32 against symbol `npy_halfbits_to_floatbits' can not be used
when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
/usr/bin/ld: numpy/core/lib/libnpymath.a(halffloat.c.12.o): relocation
R_X86_64_PC32 against symbol `npy_halfbits_to_floatbits' can not be used
when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
```

No idea, so, let's try distutils

export LAPACK=~/.local/lib/libopenblas.a
export BLAS=~/.local/lib/libopenblas.a
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.local/lib/
echo $LD_LIBRARY_PATH
```
:/usr/local/lib64/R/bin:/home/skipper/.local/lib/
```

This step seems to be necessary?

python setup.py config
```
Running from numpy source directory.
non-existing path in 'numpy/distutils': 'site.cfg'
F2PY Version 2
numpy/core/setup_common.py:88: MismatchCAPIWarning: API mismatch detected,
the C API version numbers have to be updated. Current C api version is 8,
with checksum f4362353e2d72f889fda0128aa015037, but recorded checksum for C
API version 8 in codegen_dir/cversions.txt is
17321775fc884de0b1eda478cd61c74b. If functions were added in the C API, you
have to update C_API_VERSION  in numpy/core/setup_common.py.
  MismatchCAPIWarning)
blas_opt_info:
blas_mkl_info:
  libraries mkl,vml,guide not found in ['/usr/local/lib64',
'/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
  NOT AVAILABLE

atlas_blas_threads_info:
Setting PTATLAS=ATLAS
  libraries ptf77blas,ptcblas,atlas not found in ['/usr/local/lib64',
'/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
  NOT AVAILABLE

atlas_blas_info:
  libraries f77blas,cblas,atlas not found in ['/usr/local/lib64',
'/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
  NOT AVAILABLE

/home/skipper/src/numpy-skipper/numpy/distutils/system_info.py:1501:
UserWarning:
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  warnings.warn(AtlasNotFoundError.__doc__)
blas_info:
Replacing _lib_names[0]=='blas' with 'openblas'
Replacing _lib_names[0]=='openblas' with 'openblas'
  FOUND:
    libraries = ['openblas']
    library_dirs = ['/home/skipper/.local/lib']
    language = f77

  FOUND:
    libraries = ['openblas']
    library_dirs = ['/home/skipper/.local/lib']
    define_macros = [('NO_ATLAS_INFO', 1)]
    language = f77

non-existing path in 'numpy/lib': 'benchmarks'
lapack_opt_info:
lapack_mkl_info:
mkl_info:
  libraries mkl,vml,guide not found in ['/usr/local/lib64',
'/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
  NOT AVAILABLE

  NOT AVAILABLE

atlas_threads_info:
Setting PTATLAS=ATLAS
  libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib64
  libraries lapack_atlas not found in /usr/local/lib64
  libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
  libraries lapack_atlas not found in /usr/local/lib
  libraries ptf77blas,ptcblas,atlas not found in /usr/lib64
  libraries lapack_atlas not found in /usr/lib64
  libraries ptf77blas,ptcblas,atlas not found in /usr/lib
  libraries lapack_atlas not found in /usr/lib
  libraries ptf77blas,ptcblas,atlas not found in /usr/lib/x86_64-linux-gnu
  libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
numpy.distutils.system_info.atlas_threads_info
  NOT AVAILABLE

atlas_info:
  libraries f77blas,cblas,atlas not found in /usr/local/lib64
  libraries lapack_atlas not found in /usr/local/lib64
  libraries f77blas,cblas,atlas not found in /usr/local/lib
  libraries lapack_atlas not found in /usr/local/lib
  libraries f77blas,cblas,atlas not found in /usr/lib64
  libraries lapack_atlas not found in /usr/lib64
  libraries f77blas,cblas,atlas not found in /usr/lib
  libraries lapack_atlas not found in /usr/lib
  libraries f77blas,cblas,atlas not found in /usr/lib/x86_64-linux-gnu
  libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
numpy.distutils.system_info.atlas_info
  NOT AVAILABLE

/home/skipper/src/numpy-skipper/numpy/distutils/system_info.py:1415:
UserWarning:
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  warnings.warn(AtlasNotFoundError.__doc__)
lapack_info:
Replacing _lib_names[0]=='lapack' with 'openblas'
Replacing _lib_names[0]=='openblas' with 'openblas'
  FOUND:
    libraries = ['openblas']
    library_dirs = ['/home/skipper/.local/lib']
    language = f77

  FOUND:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/home/skipper/.local/lib']
    define_macros = [('NO_ATLAS_INFO', 1)]
    language = f77

running config
```

python setup.py build &> build.log

Build log is here. Obviously it didn't go well, but I don't see anything to
indicate problems. Sometimes I am able to get _dotblas.so built, though I
don't know what causes it. This time I wasn't.

https://gist.github.com/jseabold/7054ba9d85eae09eb402#file-numpy_build-log

sudo python setup.py install &> install.log

https://gist.github.com/jseabold/a0f5638b65d44aeff598#file-numpy_install-log

>>> import numpy as np
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/numpy/__init__.py", line
138, in <module>
    import add_newdocs
  File "/usr/local/lib/python2.7/dist-packages/numpy/add_newdocs.py", line
13, in <module>
    from numpy.lib import add_newdoc
  File "/usr/local/lib/python2.7/dist-packages/numpy/lib/__init__.py", line
15, in <module>
    from polynomial import *
  File "/usr/local/lib/python2.7/dist-packages/numpy/lib/polynomial.py",
line 19, in <module>
    from numpy.linalg import eigvals, lstsq, inv
  File "/usr/local/lib/python2.7/dist-packages/numpy/linalg/__init__.py",
line 50, in <module>
    from linalg import *
  File "/usr/local/lib/python2.7/dist-packages/numpy/linalg/linalg.py",
line 25, in <module>
    from numpy.linalg import lapack_lite
ImportError: libopenblas.so.0: cannot open shared object file: No such file
or directory
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20130323/d58b5817/attachment.html 


More information about the NumPy-Discussion mailing list