[SciPy-user] Bundling numpy/scipy with applications

David Cournapeau david@ar.media.kyoto-u.ac...
Mon Nov 12 22:48:18 CST 2007


Michael Hearne wrote:
> I'm creating a Python application that is internal to my organization, 
> but will be installed on both Linux and Mac OS X machines.  The 
> application depends heavily on a number of "non-pure" modules (those 
> that have C/C++/FORTRAN components), like numpy, scipy, gdal, etc.
>
> What is the most "pythonic" way to bundle these types of modules 
> inside my application?
>
> I've investigated dist_utils and setuptools, and I don't see an easy 
> way with those tools to include build instructions for packages built 
> with autconf tools.
>
> Is my only recourse to write a custom install script that calls the 
> "configure;make;make install" from a shell?
>
You have two problems: building and packaging. For packaging, autoconf 
will not help you much outside providing a coherence between packages, I 
think; distutils/setuptools have packaging tools, but I don't know how 
good they are.

On Linux, the packaging system depends on the distribution: since it is 
only internal, the problem is severely simplified, since you are likely 
to target only one format (rpm, deb, etc...). On Mac OS X, you have also 
tools to package .pkg and .mpkg (which are a set of .pkg), available 
freely on apple dev website; I have not used them much, but they look 
simple and easy to use for simple command line packages.

You could hack something in distutils to call configure and so on, but I 
don't think it will be pleasant. Building packages using 
distutils/setuptools is easy, and do not need much configuration. So I 
think it is more natural to use a top level tool calling 
distutils/setuptools than to use distutils as the top level tool. As a 
pythonic way, you may consider build tools, such as scons or waf, both 
written in python.

But frankly, I would not bother too much about pythonic way: since some 
tools will be shell based anyway (autotools, packaging tools), and you 
don't have to care about windows, using make and shell may actually be 
easier.

For the build part, you may take a look at my garnumpy package: it is 
essentially a set of rules for Gnu Makefiles, and it can build numpy + 
scipy, with a configurable set of dependencies (ATLAS, NETLIB 
BLAS/LAPACK, fftw, etc....). It can build both distutils and 
autotools-based packages.

http://www.ar.media.kyoto-u.ac.jp/members/david/archives/garnumpy/garnumpy-0.4.tbz2

I used it sucessfully on linux and cygwin, so I would say it should work 
on Mac OS X without too much trouble. The only thing which will be 
likely to be a pain is fat binaries (Universal). I use it to build a 
totally self contained numpy/scipy installation, which is a first step 
toward packaging. If you think it can be useful to you, don't hesitate 
to ask questions; there is also a bzr archive if you want to have access 
to the dev history of the tool

cheers,

David


More information about the SciPy-user mailing list