[SciPy-dev] weave.inline broken by setuptools?

Robert Kern robert.kern@gmail....
Thu Sep 25 23:34:45 CDT 2008

On Thu, Sep 25, 2008 at 23:25, Fernando Perez <fperez.net@gmail.com> wrote:
> Howdy,
> On Thu, Sep 25, 2008 at 9:03 PM, Robert Kern <robert.kern@gmail.com> wrote:
>> It's not a workaround; it's just the correct thing to do. You must
>> always import setuptools before you import numpy.distutils.
>> numpy.distutils knows about setuptools, but setuptools doesn't know
>> about numpy.distutils, so the only way that the two get merged
>> correctly is by importing setuptools first so numpy.distutils knows to
>> merge the two.
> Well, the problem is that I didn't even know that I was importing
> numpy.distutils and that it had a non-commutative behavior with
> setuptools.  I had test code that  checked for weave's presence and
> for setuptools  and they happened to be in that order.  I could easily
> see someone doing an
> import scipy.weave
> and then
> import something.else # that unbeknownst to them, imports setuptools
> and boom.  The weirdest bugs.  Especially hard because if they happen
> to  be loading inlined code that had already been compiled before
> setuptools showed up, it will get picked up off the module cache just
> fine.  This is what happened to me: the bug only appeared on student's
> computers, because as I built the test suite I got the cache primed,
> and only later did I add the setuptools loading.
> I wonder  if numpy.distutils couldn't do a
> try: import setuptools
> except ImportError: pass
> since otherwise we basically get a ticking time bomb.  On the other
> hand, I imagine this might have its own nasty problems, so perhaps
> it's a dumb idea.

You personally asked us not to do that. That meant that you would
always be using setuptools to install numpy just because you had it on
your system rather than because you chose to use it to install numpy.

The appropriate fix is to document that weave uses numpy.distutils and
that if one is going to use both setuptools and numpy.distutils that
setuptools must be imported first. I believe you have commit access.

> This level of hidden magic/side effects really does bother me a lot.
> There may be not much we can do about it, so I guess I'll just keep on
> rasing my alertness regarding setuptools.
> Basically if your python installation starts behaving inexplicably,
> chances are very, very good that the issue has something to do with
> setuptools.

<shrug> It equally has something to do with numpy.distutils.

> At least that's been my (by now very long running)
> story....  Here's one hoping that the fork will eventually produce a
> viable long-term solution to this horrid mess.

Nope. This is intrinsic to the design of distutils. The only way to
extend the commands is to subclass. Multiple extensions at the same
time need to be merged manually. If they are implemented without
knowledge of the other, it doesn't matter how good or careful the
authors of each are. In fact, the fork just made things a little
worse. Now we have to check for both distribute and setuptools.

Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco

More information about the Scipy-dev mailing list