[SciPy-user] Bundling numpy/scipy with applications
Mon Nov 12 22:48:18 CST 2007
Michael Hearne wrote:
> I'm creating a Python application that is internal to my organization,
> but will be installed on both Linux and Mac OS X machines. The
> application depends heavily on a number of "non-pure" modules (those
> that have C/C++/FORTRAN components), like numpy, scipy, gdal, etc.
> What is the most "pythonic" way to bundle these types of modules
> inside my application?
> I've investigated dist_utils and setuptools, and I don't see an easy
> way with those tools to include build instructions for packages built
> with autconf tools.
> Is my only recourse to write a custom install script that calls the
> "configure;make;make install" from a shell?
You have two problems: building and packaging. For packaging, autoconf
will not help you much outside providing a coherence between packages, I
think; distutils/setuptools have packaging tools, but I don't know how
good they are.
On Linux, the packaging system depends on the distribution: since it is
only internal, the problem is severely simplified, since you are likely
to target only one format (rpm, deb, etc...). On Mac OS X, you have also
tools to package .pkg and .mpkg (which are a set of .pkg), available
freely on apple dev website; I have not used them much, but they look
simple and easy to use for simple command line packages.
You could hack something in distutils to call configure and so on, but I
don't think it will be pleasant. Building packages using
distutils/setuptools is easy, and do not need much configuration. So I
think it is more natural to use a top level tool calling
distutils/setuptools than to use distutils as the top level tool. As a
pythonic way, you may consider build tools, such as scons or waf, both
written in python.
But frankly, I would not bother too much about pythonic way: since some
tools will be shell based anyway (autotools, packaging tools), and you
don't have to care about windows, using make and shell may actually be
For the build part, you may take a look at my garnumpy package: it is
essentially a set of rules for Gnu Makefiles, and it can build numpy +
scipy, with a configurable set of dependencies (ATLAS, NETLIB
BLAS/LAPACK, fftw, etc....). It can build both distutils and
I used it sucessfully on linux and cygwin, so I would say it should work
on Mac OS X without too much trouble. The only thing which will be
likely to be a pain is fat binaries (Universal). I use it to build a
totally self contained numpy/scipy installation, which is a first step
toward packaging. If you think it can be useful to you, don't hesitate
to ask questions; there is also a bzr archive if you want to have access
to the dev history of the tool
More information about the SciPy-user