[SciPy-user] ***[Possible UCE]*** Re: Building Python, Numpy and Scipy with Intel compilers and MKL
Thu Apr 23 11:40:08 CDT 2009
Matthieu and David--thank you for your replies.
Today I went back and did two installs in different locations, one with icc and one with gcc. Looking at the two side-by-side, I noticed something I didn't catch yesterday:
Here's the startup message from Python built with gcc:
Python 2.5.4 (r254:67916, Apr 22 2009, 15:52:10)
[GCC 4.1.1 20070105 (Red Hat 4.1.1-52)] on linux2
Here's the startup message from Python built with icc:
Python 2.5.4 (r254:67916, Apr 23 2009, 10:48:58)
[GCC Intel(R) C++ gcc 4.1 mode] on linux2
I didn't notice the difference until I had them side-by-side (in fact, I pasted the wrong one into my first email). The Python I built with icc is, in fact, built with icc. So, that solves the mystery of the Python interpreter.
Numpy (1.3.0) works fine, whether I compile it with gcc or icc. Here is a snippet of output to show that it is actually being built with icc and linked with MKL 10.0.1:
icc -O2 -fPIC -shared build/temp.linux-x86_64-2.5/numpy/linalg/lapack_litemodule.o build/temp.linux-x86_64-2.5/numpy/linalg/python_xerbla.o -L/opt/intel/mkl/10.0.1.014/lib/em64t/ -Lbuild/temp.linux-x86_64-2.5 -lmkl_lapack -lmkl -lguide -lpthread -o build/lib.linux-x86_64-2.5/numpy/linalg/lapack_lite.so
Scipy (0.7.0) is giving me a compilation error today when I attempt to build with icc, which is different from yesterday. I had to modify numpy/distutiles/fcompiler/intel.py to manually set the version to 10.1, and I commented out all the options in method get_flags_arch which are deprecated or obsolete for modern intel compilers. The Scipy build proceeds until it tries to build scipy.interpolate.dfitpack. Here is the error I get:
icc -O2 -fPIC -shared build/temp.linux-x86_64-2.5/scipy/interpolate/src/_fitpackmodule.o -Lbuild/temp.linux-x86_64-2.5 -lfitpack -o build/lib.linux-x86_64-2.5/scipy/interpolate/_fitpack.so
building 'scipy.interpolate.dfitpack' extension
error: extension 'scipy.interpolate.dfitpack' has Fortran sources but no Fortran compiler found
Further, there are several warnings like this, and I don't know how important they are:
building 'scipy.fftpack._fftpack' extension
warning: build_ext: extension 'scipy.fftpack._fftpack' has Fortran libraries but no Fortran linker found, using default linker
I have saved the entire output of all these builds, but the Scipy build log output is about 300k and I'm open to suggestions as to the best way to share it. For starters, here is a snippet showing the compiler configuration:
building 'statlib' library
compiling Fortran sources
Fortran f77 compiler: /apps/intel/fce/10.1.008/bin/ifort -72 -w90 -w95 -fPIC -cm -O3 -unroll
Fortran f90 compiler: /apps/intel/fce/10.1.008/bin/ifort -FR -fPIC -cm -O3 -unroll
Fortran fix compiler: /apps/intel/fce/10.1.008/bin/ifort -FI -fPIC -cm -O3 -unroll
compile options: '-I/home/me/Python_Test/lib/python/numpy/core/include -c'
scipy/stats/statlib/swilk.f(85): (col. 15) remark: LOOP WAS VECTORIZED.
scipy/stats/statlib/swilk.f(120): (col. 10) remark: PARTIAL LOOP WAS VECTORIZED.
scipy/stats/statlib/spearman.f(52): (col. 12) remark: LOOP WAS VECTORIZED.
Further suggestions would be appreciated.
----- Original Message ----
From: Matthieu Brucher <firstname.lastname@example.org>
To: Craig Finch <email@example.com>; SciPy Users List <firstname.lastname@example.org>
Sent: Thursday, April 23, 2009 3:38:53 AM
Subject: ***[Possible UCE]*** Re: [SciPy-user] Building Python, Numpy and Scipy with Intel compilers and MKL
I've done the same on my configuration at work, and I didn't have a
glitch (once I got the appropriate MKL version, don't forget that any
MKL > 10.0.2 will not work). Don't forget to install your new Python
in a local folder and then to export this local/bin in your PATH and
local/lib in LD_LIBRARY_PATH.
2009/4/23 Craig Finch <email@example.com>:
> I'm trying to build Python, Numpy, and SciPy on a Red Hat system using the Intel Fortran, C, and C++ compilers, along with the Intel Math Kernel Library. This is proving to be difficult--I've tried everything I could find on the web, and it's still not working (everything works fine if I use GCC). I would really appreciate any suggestions.
> I can get Python and Numpy working correctly after compiling with Intel compilers, but I can't import anything from SciPy without getting an "undefined symbol" error. According to what I've read, this can happen when gfortran is used to compile some library (such as BLAS) while Intel fortran is used to compile SciPy. I don't think this is my problem, because I'm only using the Intel Math Kernel Libraries. Is this correct?
> Here's my latest attempt at installation:
> Python install:
> CXX='icc' LDFLAGS='-L/opt/intel/cce/10.1.008/lib' CFLAGS='-I/opt/intel/cce/10.1.008/include' CXXFLAGS='-I/opt/intel/cce/10.1.008/include' ./configure -prefix=/home/me
> Numpy install:
> python setup.py config --compiler=intel --fcompiler=intel install --home=/home/me
> Scipy install:
> setup.py config --compiler=intel --fcompiler=intel build_ext
> --fcompiler=intel -L/opt/intel/cce/10.1.008/lib install --home=/home/me
> Here's something that bothers me: when I start my Python interpreter which was compiled with icc, this is what I see:
> Python 2.5.4 (r254:67916, Apr 22 2009, 15:52:10)
> [GCC 4.1.1 20070105 (Red Hat 4.1.1-52)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> does the interpreter think it's built with GCC? Can that be causing
> problems with SciPy? How can I ensure that only Intel libraries are
> SciPy-user mailing list
Information System Engineer, Ph.D.
Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92
More information about the SciPy-user