[Numpy-discussion] parallel compilation of numpy

David Cournapeau cournape@gmail....
Thu Feb 19 00:34:24 CST 2009


On Thu, Feb 19, 2009 at 3:24 PM, Andrew Straw <strawman@astraw.com> wrote:
> David Cournapeau wrote:
>>> * Integration with setuptools and eggs, which enables things like
>>> namespace packages.
>>>
>>
>> This is not. eggs are not specified, and totally implementation defined.
>> I tried some time ago to  add an egg builder to scons, but I gave up.
>> And I don't think you can reuse the setuptools code, as everything is
>> coupled.
>
> It's an interesting idea to build Python package distributions without
> distutils. For pure Python installables, if all you seek better is
> distutils, the bar seems fairly low.

:) Being better than distutils is not difficult, indeed - that is if
you don't care about backward compatibility with distutils (which I
don't personally - since distutils is implementation defined, I don't
see any way to be backward compatible and be a significant improvement
at the same time).

> For compiled stuff, it still
> doesn't seem too bad. Of course, it is easy to say this without having
> tried...

There are some python build tools which do not have advanced
dependency handling (paver, etc...). But for compiled code, I don't
think it makes any sense not to be based on a dependency graph.
Specially for something like scipy. And this is a complex task - the
core waf code is really nice, and small (< 4000 LOC for the core).

> So, what do you mean by an "egg", in the context of it being hard to
> produce? An .egg zip file, an .egg directory, and/or a normal distutils
> package with an .egg-info/ sibling? Since .egg-info/  is now part of
> distutils, this should now be specified... or?

Producing something such as easy_install blabla.egg "works" (that is
as is intended to work in the setuptools world).

> [This, though, does point out a conceptual problem with setuptools for
> me -- it does a zillion things some of which I like a lot (e.g.
> installing console scripts and gui scripts, a simple plugin
> architecture) and others I don't care about as long as they don't break
> things for me (e.g. installing multiple version of packages
> side-by-side, a problem which is much more sanely solved by setting
> PYTHONPATH, or its sophisticated cousin, virtualenv). And all of these
> things go by one name "setuptools" and sometimes even "eggs", even
> though people often use those words to discuss totally different
> features. Hence my question above.]

I will refrain from speaking about setuptools :) But the above problem
is the same for distutils and setuptools, and exactly the fundamental
issue. If distutils was conceived as a set of lousely coupled modules
for tools, build, distribution, it would have been fixable. In the
case of setuptools, it was a conscious decision to force setuptools on
python world,

David


More information about the Numpy-discussion mailing list