[Numpy-discussion] Unable to building numpy with openblas using bento or distutils

Hanno Klemm klemm@phys.ethz...
Sat Mar 23 13:32:10 CDT 2013


Skipper,
this looks like a problem that I had in the bad old days with ATLAS, as well. Try compiling openblas with the -fPIC flag that used to help. 

Best of luck,
Hanno

hanno.klemm@me.com

Sent from my mobile device, please excuse my brevity. 

On 23.03.2013, at 19:19, Skipper Seabold <jsseabold@gmail.com> wrote:

> Some help on this would be greatly appreciated. It's been recommended to use OpenBlas over ATLAS, so I've been trying to build numpy with openblas and have run into a few problems.
> 
> 1) Build fails using bento master and waf 1.7.9, see below.
> 2) Distutils doesn't seem to be able to find lapack as part of atlas. I tried to skip a site.cfg and define environmental variables. No idea what I missed.
> 
> I followed instructions found scattered over the internet and only understand vaguely the issues. Maybe someone can help. I'd be happy to update the wiki with any answers. 
> 
> To truly support OpenBlas, is it maybe necessary to make some additions to numpy/distutils/system_info.py?
> 
> Thanks for having a look,
> 
> Skipper
> 
> Install OpenBlas
> -----------------------------
> git clone git://github.com/xianyi/OpenBLAS
> cd OpenBlas
> 
> Edit c_check to look for libpthreads in the right place (Kubuntu 12.10)
> 
> |4 $ git diff c_check
> ```
> diff --git a/c_check b/c_check
> index 4d82237..de0fd33 100644
> --- a/c_check
> +++ b/c_check
> @@ -241,7 +241,7 @@ print CONFFILE "#define FUNDERSCORE\t$need_fu\n" if $need_fu
>  
>  if ($os eq "LINUX") {
>      
> -    @pthread = split(/\s+/, `nm /lib/libpthread.so* | grep _pthread_create`);
> +    @pthread = split(/\s+/, `nm /lib/x86_64-linux-gnu/libpthread.so* | grep _pthread_create`);
>      
>      if ($pthread[2] ne "") {
>         print CONFFILE "#define PTHREAD_CREATE_FUNC     $pthread[2]\n";
> ```
> 
> make fc=gfortran
> make PREFIX=~/.local install
> 
> Everything looks ok, so far. 
> 
> Install NumPy
> ---------------------------
> Using numpy master
> 
> I tried to use bento master and waf 1.7.9, following instructions from David's blog
> 
> bentomaker configure --prefix=/home/skipper/.local --with-blas-lapack-libdir=/home/skipper/.local/lib --blas-lapack-type=openblas ..
> bentomaker build -j4
> 
> ```
> <snip>
> [101/104] cshlib: build/numpy/core/src/umath/umath_tests.c.11.o -> build/numpy/core/umath_tests.so                                                              
> /usr/bin/ld: numpy/core/lib/libnpymath.a(halffloat.c.12.o): relocation R_X86_64_PC32 against symbol `npy_halfbits_to_floatbits' can not be used when making a shared object; recompile with -fPIC
> /usr/bin/ld: final link failed: Bad value
> collect2: error: ld returned 1 exit status
> /usr/bin/ld: numpy/core/lib/libnpymath.a(halffloat.c.12.o): relocation R_X86_64_PC32 against symbol `npy_halfbits_to_floatbits' can not be used when making a shared object; recompile with -fPIC
> /usr/bin/ld: final link failed: Bad value
> collect2: error: ld returned 1 exit status
> ```
> 
> No idea, so, let's try distutils
> 
> export LAPACK=~/.local/lib/libopenblas.a
> export BLAS=~/.local/lib/libopenblas.a
> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.local/lib/
> echo $LD_LIBRARY_PATH
> ```
> :/usr/local/lib64/R/bin:/home/skipper/.local/lib/
> ```
> 
> This step seems to be necessary?
> 
> python setup.py config
> ```
> Running from numpy source directory.
> non-existing path in 'numpy/distutils': 'site.cfg'
> F2PY Version 2
> numpy/core/setup_common.py:88: MismatchCAPIWarning: API mismatch detected, the C API version numbers have to be updated. Current C api version is 8, with checksum f4362353e2d72f889fda0128aa015037, but recorded checksum for C API version 8 in codegen_dir/cversions.txt is 17321775fc884de0b1eda478cd61c74b. If functions were added in the C API, you have to update C_API_VERSION  in numpy/core/setup_common.py.
>   MismatchCAPIWarning)
> blas_opt_info:
> blas_mkl_info:
>   libraries mkl,vml,guide not found in ['/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
>   NOT AVAILABLE
> 
> atlas_blas_threads_info:
> Setting PTATLAS=ATLAS
>   libraries ptf77blas,ptcblas,atlas not found in ['/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
>   NOT AVAILABLE
> 
> atlas_blas_info:
>   libraries f77blas,cblas,atlas not found in ['/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
>   NOT AVAILABLE
> 
> /home/skipper/src/numpy-skipper/numpy/distutils/system_info.py:1501: UserWarning: 
>     Atlas (http://math-atlas.sourceforge.net/) libraries not found.
>     Directories to search for the libraries can be specified in the
>     numpy/distutils/site.cfg file (section [atlas]) or by setting
>     the ATLAS environment variable.
>   warnings.warn(AtlasNotFoundError.__doc__)
> blas_info:
> Replacing _lib_names[0]=='blas' with 'openblas'
> Replacing _lib_names[0]=='openblas' with 'openblas'
>   FOUND:
>     libraries = ['openblas']
>     library_dirs = ['/home/skipper/.local/lib']
>     language = f77
> 
>   FOUND:
>     libraries = ['openblas']
>     library_dirs = ['/home/skipper/.local/lib']
>     define_macros = [('NO_ATLAS_INFO', 1)]
>     language = f77
> 
> non-existing path in 'numpy/lib': 'benchmarks'
> lapack_opt_info:
> lapack_mkl_info:
> mkl_info:
>   libraries mkl,vml,guide not found in ['/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
>   NOT AVAILABLE
> 
>   NOT AVAILABLE
> 
> atlas_threads_info:
> Setting PTATLAS=ATLAS
>   libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib64
>   libraries lapack_atlas not found in /usr/local/lib64
>   libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
>   libraries lapack_atlas not found in /usr/local/lib
>   libraries ptf77blas,ptcblas,atlas not found in /usr/lib64
>   libraries lapack_atlas not found in /usr/lib64
>   libraries ptf77blas,ptcblas,atlas not found in /usr/lib
>   libraries lapack_atlas not found in /usr/lib
>   libraries ptf77blas,ptcblas,atlas not found in /usr/lib/x86_64-linux-gnu
>   libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
> numpy.distutils.system_info.atlas_threads_info
>   NOT AVAILABLE
> 
> atlas_info:
>   libraries f77blas,cblas,atlas not found in /usr/local/lib64
>   libraries lapack_atlas not found in /usr/local/lib64
>   libraries f77blas,cblas,atlas not found in /usr/local/lib
>   libraries lapack_atlas not found in /usr/local/lib
>   libraries f77blas,cblas,atlas not found in /usr/lib64
>   libraries lapack_atlas not found in /usr/lib64
>   libraries f77blas,cblas,atlas not found in /usr/lib
>   libraries lapack_atlas not found in /usr/lib
>   libraries f77blas,cblas,atlas not found in /usr/lib/x86_64-linux-gnu
>   libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
> numpy.distutils.system_info.atlas_info
>   NOT AVAILABLE
> 
> /home/skipper/src/numpy-skipper/numpy/distutils/system_info.py:1415: UserWarning: 
>     Atlas (http://math-atlas.sourceforge.net/) libraries not found.
>     Directories to search for the libraries can be specified in the
>     numpy/distutils/site.cfg file (section [atlas]) or by setting
>     the ATLAS environment variable.
>   warnings.warn(AtlasNotFoundError.__doc__)
> lapack_info:
> Replacing _lib_names[0]=='lapack' with 'openblas'
> Replacing _lib_names[0]=='openblas' with 'openblas'
>   FOUND:
>     libraries = ['openblas']
>     library_dirs = ['/home/skipper/.local/lib']
>     language = f77
> 
>   FOUND:
>     libraries = ['openblas', 'openblas']
>     library_dirs = ['/home/skipper/.local/lib']
>     define_macros = [('NO_ATLAS_INFO', 1)]
>     language = f77
> 
> running config
> ```
> 
> python setup.py build &> build.log
> 
> Build log is here. Obviously it didn't go well, but I don't see anything to indicate problems. Sometimes I am able to get _dotblas.so built, though I don't know what causes it. This time I wasn't.
> 
> https://gist.github.com/jseabold/7054ba9d85eae09eb402#file-numpy_build-log
> 
> sudo python setup.py install &> install.log
> 
> https://gist.github.com/jseabold/a0f5638b65d44aeff598#file-numpy_install-log
> 
> >>> import numpy as np
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/usr/local/lib/python2.7/dist-packages/numpy/__init__.py", line 138, in <module>
>     import add_newdocs
>   File "/usr/local/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 13, in <module>
>     from numpy.lib import add_newdoc
>   File "/usr/local/lib/python2.7/dist-packages/numpy/lib/__init__.py", line 15, in <module>
>     from polynomial import *
>   File "/usr/local/lib/python2.7/dist-packages/numpy/lib/polynomial.py", line 19, in <module>
>     from numpy.linalg import eigvals, lstsq, inv
>   File "/usr/local/lib/python2.7/dist-packages/numpy/linalg/__init__.py", line 50, in <module>
>     from linalg import *
>   File "/usr/local/lib/python2.7/dist-packages/numpy/linalg/linalg.py", line 25, in <module>
>     from numpy.linalg import lapack_lite
> ImportError: libopenblas.so.0: cannot open shared object file: No such file or directory
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20130323/f84052bc/attachment-0001.html 


More information about the NumPy-Discussion mailing list