[SciPy-user] Installation on dual opteron ...

Arnd Baecker arnd.baecker at web.de
Tue Sep 27 02:06:04 CDT 2005


Hi,

many thanks for your reply!

On Fri, 23 Sep 2005, Stephen Walton wrote:

[...]

> I would strongly recommend 3.4.4 and g77.  Since gfortran is part of the
> gcc4 suite, I'm not sure what will happen if you use gcc 3.x but
> gfortran instead of g77.

So I went for 3.4.4 and get stuck with the -fPIC stuff.
Presumably I am missing something simple - any help is appreciated
(list of questions below ...).

`python setup.py install` for scipy leads to:

[...]
building 'scipy.lib.lapack.flapack' extension
compiling C sources
gcc options: '-pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall
-Wstrict-prototypes -fPIC -fPIC'
compile options: '-DATLAS_INFO="\"3.7.11\"" -DNUMERIC_VERSION="\"24.0b2\""
-DNUMERIC -I/scr/python3/lib -I/scr/python3/include/python2.4 -Ibuild/src
-I/scr/python3/include/python2.4 -c'
/scr/python3/bin/g77 -shared
build/temp.linux-x86_64-2.4/build/src/build/src/Lib/lib/lapack/flapackmodule.o
build/temp.linux-x86_64-2.4/build/src/fortranobject.o -L/scr/python3/lib64
-Lbuild/temp.linux-x86_64-2.4 -llapack -lptf77blas -lptcblas -latlas -lg2c
-o build/lib.linux-x86_64-2.4/scipy/lib/lapack/flapack.so
/usr/bin/ld: /scr/python3/lib64/liblapack.a(slamch.o): relocation
R_X86_64_32S against `a local symbol' can not be used when making a shared
object; recompile with -fPIC
/scr/python3/lib64/liblapack.a: could not read symbols: Bad value
collect2: ld returned 1 exit status
error: Command "/scr/python3/bin/g77 -shared
build/temp.linux-x86_64-2.4/build/src/build/src/Lib/lib/lapack/flapackmodule.o
build/temp.linux-x86_64-2.4/build/src/fortranobject.o -L/scr/python3/lib64
-Lbuild/temp.linux-x86_64-2.4 -llapack -lptf77blas -lptcblas -latlas -lg2c
-o build/lib.linux-x86_64-2.4/scipy/lib/lapack/flapack.so" failed with
exit status 1/usr/bin/ld: /scr/python3/lib64/liblapack.a(slamch.o):
relocation R_X86_64_32S against `a local symbol' can not be used when
making a shared object; recompile with -fPIC
/scr/python3/lib64/liblapack.a: could not read symbols: Bad value
collect2: ld returned 1 exit status


- I compiled atlas etc. with -fPIC
  (at least I tried to enforce that)
- Is there a way to check the resulting libraries if they were compiled
  properly with -fPIC?
- It *seems* that Numeric works with the ATLAS I compiled
- Is there a way to check (after installation)
  against which libraries Numeric and scipy are linked
  (does it use ATLAS, does it use dotblas?).
- Below I attach my notes of the steps of the installation.
  Maybe some expert can spot what I am doing wrong...

Many thanks in advance,

Arnd


=============
Install notes
=============

Directory ``Sources``
=====================

gcc-3.4.4.tar.bz2
Python-2.4.1.tar.bz2
ipython.zip            (from CVS)
atlas3.7.11.tar.bz2
Numerical.zip          (from CVS)
f2py2e.zip             (from CVS)
scipy.zip              (from svn, includes scipy_core)



Environment variables
======================

export PHOME=/scr/python3/
export         INFODIR=${PHOME}/info:$INFODIR
export        INFOPATH=${PHOME}/info:$INFOPATH
export         MANPATH=${PHOME}/man:$MANPATH
export LD_LIBRARY_PATH=${PHOME}/lib64:${PHOME}/lib:$LD_LIBRARY_PATH
export     LD_RUN_PATH=${PHOME}/lib64:${PHOME}/lib:$LD_RUN_PATH

export PYTHONDOCS=${PHOME}/docs/Python-Docs-2.4.1/
export PATH=${PHOME}/bin:${PATH}
export PAGER="less -R"

export CFLAGS=-fPIC

For the installation::

  mkdir CompileDir
  cd CompileDir


gcc
===


tar xjf ../Sources/gcc-3.4.4.tar.bz2
mkdir gcc-build
cd gcc-build

../gcc-3.4.4/configure --prefix=${PHOME} --enable-shared
--enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu
--enable-languages=c,c++,f77,objc

make
make install

cd /scr/python3/bin/
ln -s gcc /scr/python3/bin/cc
cd -


Python
======

tar xjf ../Sources/Python-2.4.1.tar.bz2
cd Python-2.4.1
./configure --prefix=${PHOME}

make
make install

make test

255 tests OK.
35 tests skipped:
    test_aepack test_al test_applesingle test_bsddb185 test_bsddb3
    test_cd test_cl test_codecmaps_cn test_codecmaps_hk
    test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
    test_dl test_gl test_imageop test_imgfile test_linuxaudiodev
    test_macfs test_macostools test_nis test_normalization
    test_ossaudiodev test_pep277 test_plistlib test_rgbimg
    test_scriptpackages test_socket_ssl test_socketserver
    test_sunaudiodev test_timeout test_urllib2net test_urllibnet
    test_winreg test_winsound
Those skips are all expected on linux2.

cd ..



IPython
=======

unzip ../Sources/ipython.zip
cd ipython
python setup.py install
cd ..


ATLAS and BLAS/LAPACK
=====================

See:
http://www.scipy.org/documentation/buildatlas4scipy.txt


export BLAS_SRC=${HOME}/INSTALL_PYTHON/CompileDir/BLAS
export LAPACK_SRC=${HOME}/INSTALL_PYTHON/CompileDir/LAPACK

mkdir -p $BLAS_SRC
cd $BLAS_SRC
wget http://www.netlib.org/blas/blas.tgz
tar xzf blas.tgz

g77 -fno-second-underscore -O2  -fPIC -c *.f


ar r libfblas.a *.o
ranlib libfblas.a
rm -rf *.o

mkdir -p $LAPACK_SRC
cd $LAPACK_SRC/..
wget http://www.netlib.org/lapack/lapack.tgz
tar xzf lapack.tgz
cd $LAPACK_SRC


cp INSTALL/make.inc.LINUX make.inc             # on LINUX
# In OPTS variable replace '-fno-f2c -O3' with '-O2 -fPIC'
make lapacklib

make clean
cd ..



----------

export BLAS=$BLAS_SRC/libfblas.a


tar xjf ../Sources/atlas3.7.11.tar.bz2
cd ATLAS


make xconfig
   ./xconfig -F f "-fomit-frame-pointer -O -fno-second-underscore -fPIC" \
      -F c " -fomit-frame-pointer -O -mfpmath=387 -m64 -fPIC"  \
      -F m "-fomit-frame-pointer -O -mfpmath=387 -m64 -fPIC"   -b $BLAS

 - 64 Bit
 - Posix threads
 - don't stop
 - use express setup? [y]:

 -  Enter f77 compiler [g77]: gfortran
    Enter F77 Flags [-fomit-frame-pointer -O -m64]:
 - Tune the Level 1 BLAS? [y]:

# Gives for the compile commands:
F77 = /scr/python3//bin/g77 -fomit-frame-pointer -O -fno-second-underscore
-fPIC
CC = /scr/python3//bin/gcc -fomit-frame-pointer -O -mfpmath=387 -m64 -fPIC
MCC = /scr/python3//bin/gcc -fomit-frame-pointer -O -mfpmath=387 -m64
-fPIC


make sanity_test arch=Linux_HAMMER64SSE3_2
make ptsanity_test arch=Linux_HAMMER64SSE3_2
# both work fine


* Make optimized LAPACK library::

       cd lib//Linux_HAMMER64SSE3_2/
       mkdir tmp; cd tmp
       ar x ../liblapack.a
       cp ../liblapack.a ../liblapack.a_SAVE
       cp ../../../../LAPACK/lapack_LINUX.a ../liblapack.a
       ar r ../liblapack.a *.o
       cd ..; rm -rf tmp

ls -lh liblapack.a
-rw-r--r--  1 arnd users 10M 2005-09-26 14:11 liblapack.a

cp *.a $PHOME/lib64
cd ../../
cp include/{cblas.h,clapack.h} $PHOME/include


Numerical
=========

unzip ../Sources/Numerical.zip
cd Numerical

emacs customize.py
if 1:
    use_system_lapack = 1
    lapack_library_dirs = ['/scr/python3//lib64/']
    lapack_libraries = ['lapack', 'cblas', 'f77blas', 'atlas', 'g2c']

if 1:
    use_dotblas = 1
    dotblas_include_dirs = ['/scr/python3/include/atlas']
    dotblas_library_dirs = lapack_library_dirs
    dotblas_libraries = ['cblas', 'atlas', 'g2c']


python setup.py build  | tee ../Numeric_install_log.txt
python setup.py install  | tee ../Numeric_install_log.txt
cd ..


scipy
=====

# Fortran to Python
unzip ../Sources/f2py2e.zip
cd  f2py2e
make install
cd ..


# scipy:
unzip ../Sources/scipy.zip
cd scipy


export ATLAS=/scr/python3/lib64:/scr/python3/include

cd scipy_core/scipy_distutils
python setup.py install
cd ../..

# scipy_systeminfo.txt for report on installed packages
python scipy_core/scipy_distutils/system_info.py > ../scipy_systeminfo.txt

python setup.py install | tee ../scipy_install_log.txt


I also tried:

python setup.py config_fc --opt "-fPIC" build  | tee ../scipy_install_log.txt

but no difference ...





More information about the SciPy-user mailing list