[Numpy-discussion] Compiling numpy using icc gets missing library error

[Hoyt Koepke]

Hello,

I've been trying out intel's compiler with python and so far things
have been going pretty well.  I managed to compile python 2.7.1 with
icc (with help from
http://software.intel.com/en-us/forums/showthread.php?t=56652), so
distutils automatically defaults to the icc compiler.  So far a number
of packages work great, so the overall system seems to work fine.

However, I'm running into a few problems compiling numpy.  I'm using
the latest development build from git (rev ac3cba). The first is that
some typing for long double isn't getting recognized -- building numpy
breaks with the following error (I'll also post a bug report on this
if requested).

Traceback (most recent call last):
  File "setup.py", line 201, in <module>
    setup_package()
  File "setup.py", line 194, in setup_package
    configuration=configuration )
  File "/home/hoytak/sysroot/src/numpy/numpy/distutils/core.py", line
186, in setup
    return old_setup(**new_attr)
  File "/home/hoytak/sysroot/lib/python2.7/distutils/core.py", line
152, in setup
    dist.run_commands()
  File "/home/hoytak/sysroot/lib/python2.7/distutils/dist.py", line
953, in run_commands
    self.run_command(cmd)
  File "/home/hoytak/sysroot/lib/python2.7/distutils/dist.py", line
972, in run_command
    cmd_obj.run()
  File "/home/hoytak/sysroot/src/numpy/numpy/distutils/command/build.py",
line 37, in run
    old_build.run(self)
  File "/home/hoytak/sysroot/lib/python2.7/distutils/command/build.py",
line 127, in run
    self.run_command(cmd_name)
  File "/home/hoytak/sysroot/lib/python2.7/distutils/cmd.py", line
326, in run_command
    self.distribution.run_command(command)
  File "/home/hoytak/sysroot/lib/python2.7/distutils/dist.py", line
972, in run_command
    cmd_obj.run()
  File "/home/hoytak/sysroot/src/numpy/numpy/distutils/command/build_src.py",
line 152, in run
    self.build_sources()
  File "/home/hoytak/sysroot/src/numpy/numpy/distutils/command/build_src.py",
line 169, in build_sources
    self.build_extension_sources(ext)
  File "/home/hoytak/sysroot/src/numpy/numpy/distutils/command/build_src.py",
line 328, in build_extension_sources
    sources = self.generate_sources(sources, ext)
  File "/home/hoytak/sysroot/src/numpy/numpy/distutils/command/build_src.py",
line 385, in generate_sources
    source = func(extension, build_dir)
  File "numpy/core/setup.py", line 442, in generate_config_h
    rep = check_long_double_representation(config_cmd)
  File "numpy/core/setup_common.py", line 136, in
check_long_double_representation
    type = long_double_representation(pyod(object))
  File "numpy/core/setup_common.py", line 280, in long_double_representation
    raise ValueError("Could not lock sequences (%s)" % saw)
ValueError: Could not lock sequences (None)

This happens regardless of whether I use the --compiler=intel option.
I'm using one of the new sandy bridge processors (i7-2600k), which may
be part of the reason.

Looking at the code, it seems this comes from a failure to detect the
type info from a particular string.  I've attached a dump of the lines
variable passed to the long_double_representation function.  As a
workaround, I set it manually to return 'INTEL_EXTENDED_12_BYTES_LE'
with the idea that I could test this using the unit tests later on.
When I did that, everything else compiled fine and numpy installed.
However, attempting to import numpy gives me this:

>>> import numpy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/hoytak/sysroot/lib/python2.7/site-packages/numpy/__init__.py",
line 137, in <module>
    import add_newdocs
  File "/home/hoytak/sysroot/lib/python2.7/site-packages/numpy/add_newdocs.py",
line 9, in <module>
    from numpy.lib import add_newdoc
  File "/home/hoytak/sysroot/lib/python2.7/site-packages/numpy/lib/__init__.py",
line 4, in <module>
    from type_check import *
  File "/home/hoytak/sysroot/lib/python2.7/site-packages/numpy/lib/type_check.py",
line 8, in <module>
    import numpy.core.numeric as _nx
  File "/home/hoytak/sysroot/lib/python2.7/site-packages/numpy/core/__init__.py",
line 5, in <module>
    import multiarray
ImportError: /home/hoytak/sysroot/lib/python2.7/site-packages/numpy/core/multiarray.so:
undefined symbol: npy_half_to_float

In case it's relevant:

$ ldd /home/hoytak/sysroot/lib/python2.7/site-packages/numpy/core/multiarray.so
        linux-vdso.so.1 =>  (0x00007fff60fff000)
        libpython2.7.so.1.0 =>
/home/hoytak/sysroot/lib/libpython2.7.so.1.0 (0x00007f5916dc3000)
        libimf.so =>
/usr/intel/composerxe-2011.2.137/composerxe-2011.2.137/compiler/lib/intel64/libimf.so
(0x00007f59169df000)
        libsvml.so =>
/usr/intel/composerxe-2011.2.137/composerxe-2011.2.137/compiler/lib/intel64/libsvml.so
(0x00007f5916337000)
        libm.so.6 => /lib/libm.so.6 (0x00007f5916093000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f5915e7c000)
        libintlc.so.5 =>
/usr/intel/composerxe-2011.2.137/composerxe-2011.2.137/compiler/lib/intel64/libintlc.so.5
(0x00007f5915d2d000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x00007f5915b10000)
        libc.so.6 => /lib/libc.so.6 (0x00007f591578c000)
        libdl.so.2 => /lib/libdl.so.2 (0x00007f5915588000)
        libutil.so.1 => /lib/libutil.so.1 (0x00007f5915385000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f591770c000)


I also attached the build log here (which is the same for the first
error up to the crash).  Any suggestions of where to look or where
something could have gone wrong?

Thanks!
-- Hoyt

++++++++++++++++++++++++++++++++++++++++++++++++
+ Hoyt Koepke
+ University of Washington Department of Statistics
+ http://www.stat.washington.edu/~hoytak/
+ hoytak@gmail.com
++++++++++++++++++++++++++++++++++++++++++
-------------- next part --------------
A non-text attachment was scrubbed...
Name: build.log.gz
Type: application/x-gzip
Size: 10983 bytes
Desc: not available
Url : http://mail.scipy.org/pipermail/numpy-discussion/attachments/20110322/af1302fb/attachment-0002.gz 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lines_dump.p.gz
Type: application/x-gzip
Size: 3385 bytes
Desc: not available
Url : http://mail.scipy.org/pipermail/numpy-discussion/attachments/20110322/af1302fb/attachment-0003.gz