[Numpy-discussion] Problem with numpy and distutils on OS X

Robert Kern robert.kern@gmail....
Fri May 9 13:35:38 CDT 2008


On Fri, May 9, 2008 at 11:17 AM, Michele Vallisneri
<vallis.35530053@bloglines.com> wrote:
> I'm writing a standard distutils setup.py to compile a Python C
> extension on OS X 10.4, and I need to specify a few special compiler
> options to enable vector CPU extension (altivec and SSE on i686 and
> PPC respectively). This compromises the generation of universal
> binaries, because these options are CPU-specific, and cannot be
> passed to gcc together with "-arch ppc -arch i386".
>
> I'm happy with generating a nonuniversal extension, which I can do
> (at least with distutils 2.4.4) by specifying an extra "-arch i386"
> compiler option. Distutils has some Darwin-specific code that catches
> that, and takes out the "-arch ppc" option that would be there by
> default.
>
> All well, but then in my setup.py I also import numpy (to find out
> where its C includes are). If I do that, the behavior of distutils
> changes, and the "-arch ppc" option is not taken out, so compilation
> fails again.
>
> So I have questions:
>
> - Is this an acceptable behavior for numpy to have?

No. We've seen this before, and I thought we fixed it, but perhaps
not. numpy.distutils does monkeypatch distutils, but you shouldn't get
that unless if you import numpy.distutils.

> Should it modify
> other modules? I understand that there is a numpy.distutils, but
> shouldn't I have a choice to use it explicitly, and get the old
> distutils behavior by using that namespace?

Yes, you should be able to avoid this by avoiding importing
numpy.distutils. However, I don't see why you are getting this. Just
importing numpy and calling numpy.get_include() does not bring in
numpy.distutils, at least not with the SVN version of numpy (the 1.1.x
branch rather than the 1.2.x trunk).

>>> import sys
>>> old = set(sys.modules)
>>> import numpy
>>> numpy.get_include()
'/Users/rkern/svn/numpy/numpy/core/include'
>>> new = set(sys.modules)
>>> [x for x in (new - old) if 'distutils' in x]
[]
>>>

You might try something similar at the top of your setup.py script to
see if 1.0.4 does something different.

> - Is there a way to avoid or disable this interference?

Possibly upgrade to the 1.1.x branch. Check it out from here:

  http://svn.scipy.org/svn/numpy/branches/1.1.x

> - Finally, is there a way to compile universal binaries with CPU-
> specific options?

Not to my knowledge, no. There might be a gcc option to use in
extra_compile_args, but you will have to check the man page for it. If
you find one, please post it, since I am interested in having such a
capability myself.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco


More information about the Numpy-discussion mailing list