[NumPy-Tickets] [NumPy] #1399: distutils fails to build ppc64 support on Mac OS X when requested

[NumPy Trac]

#1399: distutils fails to build ppc64 support on Mac OS X when requested
---------------------------------+------------------------------------------
  Reporter:  david.warde-farley  |       Owner:  rgommers                                      
      Type:  defect              |      Status:  reopened                                      
  Priority:  highest             |   Milestone:  1.5.1                                         
 Component:  numpy.distutils     |     Version:  devel                                         
Resolution:                      |    Keywords:  macosx, osx, distutils, universalbinary, ppc64
---------------------------------+------------------------------------------

Comment(by matthew.brett):

 Yes - investigating further - my previous patch enables compilation with
 the C flags architectures, but doesn't affect the .so linker args.  We
 didn't see a problem before because the default linker args were
 generating i386-only .so files.  These worked OK for an i386 python, but
 python 2.7 is x86_64 and so blows up when trying to use the i386 .so
 files.

 We can make fat (i386, ppc64, x86_64) .so files with something like this
 patch:

 {{{
 diff --git a/numpy/distutils/fcompiler/gnu.py
 b/numpy/distutils/fcompiler/gnu.py
 index a18a7a6..b149c71 100644
 --- a/numpy/distutils/fcompiler/gnu.py
 +++ b/numpy/distutils/fcompiler/gnu.py
 @@ -102,6 +102,9 @@ class GnuFCompiler(FCompiler):
      def get_flags_linker_so(self):
          opt = self.linker_so[1:]
          if sys.platform=='darwin':
 +            # Add any arch flags
 +            opt.extend(self._c_arch_flags())
 +            # Work out deployment target
              target = os.environ.get('MACOSX_DEPLOYMENT_TARGET', None)
              # If MACOSX_DEPLOYMENT_TARGET is set, we simply trust the
 value
              # and leave it alone.  But, distutils will complain if the
 }}}

 However, this leads to problems when using the 'libmach.a' library:

 {{{
 /usr/local/bin/gfortran -Wall -Wall -undefined dynamic_lookup -bundle
 -arch i386 -arch ppc -arch x86_64
 build/temp.macosx-10.5-fat3-2.7/scipy/integrate/_quadpackmodule.o
 -Lbuild/temp.macosx-10.5-fat3-2.7 -lquadpack -llinpack_lite -lmach
 -lgfortran -o build/lib.macosx-10.5-fat3-2.7/scipy/integrate/_quadpack.so
 ld: warning: object file compiled with -mlong-branch which is no longer
 needed. To remove this warning, recompile without -mlong-branch:
 /usr/local/lib/gcc/powerpc-apple-darwin8/4.2.3/crt3.o
 ld: warning: in build/temp.macosx-10.5-fat3-2.7/libmach.a, file was built
 for unsupported file format which is not the architecture being linked
 (ppc)
 ld: warning: in build/temp.macosx-10.5-fat3-2.7/libmach.a, file was built
 for unsupported file format which is not the architecture being linked
 (x86_64)
 }}}

 This in turn is because libmach.a has opts turned off by setup.py, maybe
 because it's trying to detect machine specific things, and therefore
 should not (cannot?) be used in cross-compile mode, as we are doing here?

 {{{
 building 'mach' library
 using additional config_fc from setup script for fortran compiler:
 {'noopt': ('scipy/integrate/setup.pyc', 1)}
 customize Gnu95FCompiler
 compiling Fortran sources
 Fortran f77 compiler: /usr/local/bin/gfortran -Wall -ffixed-form -fno-
 second-underscore -fPIC
 Fortran f90 compiler: /usr/local/bin/gfortran -Wall -fno-second-underscore
 -fPIC
 Fortran fix compiler: /usr/local/bin/gfortran -Wall -ffixed-form -fno-
 second-underscore -Wall -fno-second-underscore -fPIC
 }}}

 and

 {{{
 building 'sc_mach' library
 using additional config_fc from setup script for fortran compiler:
 {'noopt': ('scipy/special/setup.pyc', 1)}
 customize Gnu95FCompiler
 compiling Fortran sources
 Fortran f77 compiler: /usr/local/bin/gfortran -Wall -ffixed-form -fno-
 second-underscore -fPIC
 Fortran f90 compiler: /usr/local/bin/gfortran -Wall -fno-second-underscore
 -fPIC
 Fortran fix compiler: /usr/local/bin/gfortran -Wall -ffixed-form -fno-
 second-underscore -Wall -fno-second-underscore -fPIC
 }}}

 leading to scipy.test() errors like:

 {{{
 ======================================================================
 ERROR: Failure: ImportError
 (dlopen(/Users/mb312/.virtualenvs/python27/lib/python2.7/site-
 packages/scipy/special/_cephes.so, 2): Symbol not found: _d1mach_
   Referenced from: /Users/mb312/.virtualenvs/python27/lib/python2.7/site-
 packages/scipy/special/_cephes.so
   Expected in: dynamic lookup
 )
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "/Users/mb312/.virtualenvs/python27/lib/python2.7/site-
 packages/nose-0.11.4-py2.7.egg/nose/loader.py", line 382, in
 loadTestsFromName
     addr.filename, addr.module)
   File "/Users/mb312/.virtualenvs/python27/lib/python2.7/site-
 packages/nose-0.11.4-py2.7.egg/nose/importer.py", line 39, in
 importFromPath
     return self.importFromDir(dir_path, fqname)
   File "/Users/mb312/.virtualenvs/python27/lib/python2.7/site-
 packages/nose-0.11.4-py2.7.egg/nose/importer.py", line 86, in
 importFromDir
     mod = load_module(part_fqname, fh, filename, desc)
   File "/Users/mb312/.virtualenvs/python27/lib/python2.7/site-
 packages/scipy/stats/__init__.py", line 7, in <module>
     from stats import *
   File "/Users/mb312/.virtualenvs/python27/lib/python2.7/site-
 packages/scipy/stats/stats.py", line 202, in <module>
     import scipy.special as special
   File "/Users/mb312/.virtualenvs/python27/lib/python2.7/site-
 packages/scipy/special/__init__.py", line 8, in <module>
     from basic import *
   File "/Users/mb312/.virtualenvs/python27/lib/python2.7/site-
 packages/scipy/special/basic.py", line 6, in <module>
     from _cephes import *
 ImportError: dlopen(/Users/mb312/.virtualenvs/python27/lib/python2.7/site-
 packages/scipy/special/_cephes.so, 2): Symbol not found: _d1mach_
   Referenced from: /Users/mb312/.virtualenvs/python27/lib/python2.7/site-
 packages/scipy/special/_cephes.so
   Expected in: dynamic lookup
 }}}

 So - I suppose our only hope is that someone understands the mach .f files
 well enough to know what to do.

-- 
Ticket URL: <http://projects.scipy.org/numpy/ticket/1399#comment:17>
NumPy <http://projects.scipy.org/numpy>
My example project