[SciPy-dev] Compiling scipy with Intel ifort & MKL

rex rex@nosyntax....
Mon Jun 11 22:31:06 CDT 2007


rex <rex@nosyntax.com> [2007-06-11 16:12]:
> Pearu Peterson <pearu@cens.ioc.ee> [2007-06-11 15:12]:
> > It is due to a bug in numpy introduced, I think, just before releasing 
> > 1.0.3. It has been fixed in numpy svn. 
> 
> Thank you! Building numpy from svn did eliminate the "error: file
> 'dfftpack/*.f' does not exist" message when building scipy, and the
> scipy compilation proceeded much further. However, it still produces an
> error which appears to be due to a failure to recognize the ID of ifort
> 10.
> 
> I've already had to fix one problem in distutils due to Intel changing
> 'mkl_lapack32' in MKL8.1 to 'mkl_lapack' in MKL9.1. It appears another
> change is needed, perhaps in numpy/distutils/fcompiler/intel.py

I don't understand the distutils/* code well enough to fix it, but it
appears that the problem is triggered by packages that cause:

library 'mach' defined more than once, overwriting build_info {'sources': ['Lib/integrate/mach/i1mach.f', 'Lib/integrate/mach/d1mach.f', 'Lib/integrate/mach/r1mach.f', 'Lib/integrate/mach/xerror.f'], 'config_fc': {'noopt': ('Lib/integrate/setup.pyc', 1)}, 'source_languages': ['f77']} with {'sources': ['Lib/special/mach/i1mach.f', 'Lib/special/mach/d1mach.f', 'Lib/special/mach/r1mach.f', 'Lib/special/mach/xerror.f'], 'config_fc': {'noopt': ('Lib/special/setup.pyc', 1)}, 'source_languages': ['f77']}.
extending extension 'scipy.linsolve._zsuperlu' defined_macros with [('USE_VENDOR_BLAS', 1)]


This results in an error:

[...]
customize IntelCCompiler
customize IntelCCompiler using build_ext
Traceback (most recent call last):
  File "setup.py", line 55, in <module>
    setup_package()
  File "setup.py", line 47, in setup_package
    configuration=configuration )
  File "/usr/local/lib/python2.5/site-packages/numpy/distutils/core.py", line 176, in setup
    return old_setup(**new_attr)
  File "/usr/lib/python2.5/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/lib/python2.5/distutils/dist.py", line 974, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python2.5/distutils/dist.py", line 994, in run_command
    cmd_obj.run()
  File "/usr/lib/python2.5/distutils/command/build.py", line 112, in run
    self.run_command(cmd_name)
  File "/usr/lib/python2.5/distutils/cmd.py", line 333, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python2.5/distutils/dist.py", line 994, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python2.5/site-packages/numpy/distutils/command/build_ext.py", line 181, in run
    if fcompiler and fcompiler.get_version():
  File "/usr/local/lib/python2.5/site-packages/numpy/distutils/ccompiler.py", line 265, in CCompiler_get_version
    cmd = ' '.join(version_cmd)
TypeError: sequence item 1: expected string, NoneType found

I determined this by the brute force method of deleting the current .f file
that triggered the problem. The result was another .f file causing the
same problem, leading to my conclusion that something in distutils/* is
broken for the v10 Intel compilers. (I've already verified and fixed a
problem in distutils/* re v9.1 MKL).

Here's a shortend (by ~5000 lines) session illustrating the error:
/usr/local/src/scipy/python setup.py config --compiler=intel --fcompiler=intel build >& build_noansari.log
mkl_info:
  FOUND:
    libraries = ['mkl', 'vml', 'guide', 'pthread']
    library_dirs = ['/opt/intel/mkl/9.1/lib/32']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['/opt/intel/mkl/9.1/include']

blas_opt_info:
blas_mkl_info:
  FOUND:
    libraries = ['mkl', 'vml', 'guide', 'pthread']
    library_dirs = ['/opt/intel/mkl/9.1/lib/32']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['/opt/intel/mkl/9.1/include']

  FOUND:
    libraries = ['mkl', 'vml', 'guide', 'pthread']
    library_dirs = ['/opt/intel/mkl/9.1/lib/32']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['/opt/intel/mkl/9.1/include']

lapack_opt_info:
lapack_mkl_info:
  FOUND:
    libraries = ['mkl_lapack', 'mkl', 'vml', 'guide', 'pthread']
    library_dirs = ['/opt/intel/mkl/9.1/lib/32']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['/opt/intel/mkl/9.1/include']

  FOUND:
    libraries = ['mkl_lapack', 'mkl', 'vml', 'guide', 'pthread']
    library_dirs = ['/opt/intel/mkl/9.1/lib/32']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['/opt/intel/mkl/9.1/include']

non-existing path in 'Lib/linsolve': 'tests'
umfpack_info:
  libraries umfpack not found in /opt/intel/mkl/9.1/lib/32
/usr/local/lib/python2.5/site-packages/numpy/distutils/system_info.py:403: UserWarning: 
    UMFPACK sparse solver (http://www.cise.ufl.edu/research/sparse/umfpack/)
    not found. Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [umfpack]) or by setting
    the UMFPACK environment variable.
  warnings.warn(self.notfounderror.__doc__)
  NOT AVAILABLE

Warning: Subpackage 'Lib' configuration returned as 'scipy'
non-existing path in 'Lib/maxentropy': 'doc'
running config
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
building py_modules sources
creating build
creating build/src.linux-i686-2.5
creating build/src.linux-i686-2.5/scipy
building library "dfftpack" sources
building library "linpack_lite" sources
building library "mach" sources
building library "quadpack" sources
building library "odepack" sources
building library "fitpack" sources
building library "superlu_src" sources
building library "odrpack" sources
building library "minpack" sources
building library "rootfind" sources
building library "c_misc" sources
building library "cephes" sources
building library "mach" sources
building library "toms" sources
building library "amos" sources
building library "cdf" sources
building library "specfun" sources
building library "statlib" sources
building extension "scipy.cluster._vq" sources
building extension "scipy.fftpack._fftpack" sources
creating build/src.linux-i686-2.5/Lib
creating build/src.linux-i686-2.5/Lib/fftpack
f2py options: []
f2py: Lib/fftpack/fftpack.pyf
Reading fortran codes...
	Reading file 'Lib/fftpack/fftpack.pyf' (format:free)
Post-processing...
	Block: _fftpack
			Block: zfft
			Block: drfft
			Block: zrfft
			Block: zfftnd
			Block: destroy_zfft_cache
			Block: destroy_zfftnd_cache
			Block: destroy_drfft_cache
Post-processing (stage 2)...
Building modules...
	Building module "_fftpack"...
		Constructing wrapper function "zfft"...
getarrdims:warning: assumed shape array, using 0 instead of '*'

[snip]

Lib/fftpack/dfftpack/dfftb1.f(129): (col. 13) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftb1.f(142): (col. 16) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftb1.f(187): (col. 10) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftb1.f(214): (col. 16) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftb1.f(384): (col. 13) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftb1.f(272): (col. 13) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftb1.f(237): (col. 13) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftb1.f(315): (col. 13) remark: LOOP WAS VECTORIZED.
ifort:f77: Lib/fftpack/dfftpack/dfftf1.f
ifort: command line remark #10148: option '-K' not supported
Lib/fftpack/dfftpack/dfftf1.f(56): (col. 10) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(135): (col. 10) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(76): (col. 10) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(103): (col. 16) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(153): (col. 13) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(166): (col. 16) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(185): (col. 10) remark: PERMUTED LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(180): (col. 13) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(204): (col. 16) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(379): (col. 13) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(277): (col. 13) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(244): (col. 13) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(318): (col. 13) remark: LOOP WAS VECTORIZED.
ifort:f77: Lib/fftpack/dfftpack/dffti1.f
ifort: command line remark #10148: option '-K' not supported

[at least hundreds of yummy "LOOP WAS VECTORIZED" lines (and others) snipped]

customize IntelCCompiler
customize IntelCCompiler using build_ext
library 'mach' defined more than once, overwriting build_info {'sources': ['Lib/integrate/mach/i1mach.f', 'Lib/integrate/mach/d1mach.f', 'Lib/integrate/mach/r1mach.f', 'Lib/integrate/mach/xerror.f'], 'config_fc': {'noopt': ('Lib/integrate/setup.pyc', 1)}, 'source_languages': ['f77']} with {'sources': ['Lib/special/mach/i1mach.f', 'Lib/special/mach/d1mach.f', 'Lib/special/mach/r1mach.f', 'Lib/special/mach/xerror.f'], 'config_fc': {'noopt': ('Lib/special/setup.pyc', 1)}, 'source_languages': ['f77']}.
extending extension 'scipy.linsolve._zsuperlu' defined_macros with [('USE_VENDOR_BLAS', 1)]
extending extension 'scipy.linsolve._dsuperlu' defined_macros with [('USE_VENDOR_BLAS', 1)]
extending extension 'scipy.linsolve._csuperlu' defined_macros with [('USE_VENDOR_BLAS', 1)]
extending extension 'scipy.linsolve._ssuperlu' defined_macros with [('USE_VENDOR_BLAS', 1)]
customize IntelCCompiler
customize IntelCCompiler using build_ext
Traceback (most recent call last):
  File "setup.py", line 55, in <module>
    setup_package()
  File "setup.py", line 47, in setup_package
    configuration=configuration )
  File "/usr/local/lib/python2.5/site-packages/numpy/distutils/core.py", line 176, in setup
    return old_setup(**new_attr)
  File "/usr/lib/python2.5/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/lib/python2.5/distutils/dist.py", line 974, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python2.5/distutils/dist.py", line 994, in run_command
    cmd_obj.run()
  File "/usr/lib/python2.5/distutils/command/build.py", line 112, in run
    self.run_command(cmd_name)
  File "/usr/lib/python2.5/distutils/cmd.py", line 333, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python2.5/distutils/dist.py", line 994, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python2.5/site-packages/numpy/distutils/command/build_ext.py", line 181, in run
    if fcompiler and fcompiler.get_version():
  File "/usr/local/lib/python2.5/site-packages/numpy/distutils/ccompiler.py", line 265, in CCompiler_get_version
    cmd = ' '.join(version_cmd)
TypeError: sequence item 1: expected string, NoneType found


More information about the Scipy-dev mailing list