[SciPy-dev] Low-level code integration discussion at scipy'03?
fperez at colorado.edu
Tue Sep 2 16:28:00 CDT 2003
as I mentioned before, I think it would be great to have a bit of an open
discussion on methods and techniques for plugging low-level code into python
at Scipy'03. I don't mean to teach people how to write an extension module,
but rather what the best/easiest approaches are for writing time-critical code
which manipulates Numpy/numarray arrays.
A brief summary of the alternatives:
* Scipy's weave relies on blitz, which is extremely nice and maps surprisingly
well to python arrays. Blitz arrays have slices, ranges, overloaded
arithmetic operators and math functions, etc. In fact, writing blitz C++
feels almost like writing python code, with the caveats of handling
compile-time type information.
But blitz development seems to be rather slow these days, and the issue of
access to Blas/lapack from blitz isn't fully settled (i.e., it's not
out-of-the box easy).
* The ublas library
(http://www.boost.org/libs/numeric/ublas/doc/overview.htm). It has direct
blas support, but it seems to focus strictly on vectors and matrices, and as
far as I see it doesn't support higher rank objects (I could be wrong, though:
I've been using blitz for 'real' and only looked at the ublas webpage).
* Writing low-level code in fortran and using Pearu's f2py. I love f2py, but
I'd prefer my low-level code to have the ability to also use more information
from the python side of things if needed. I think f2py is a great tool for
wrapping existing fortran codes, but not so much for extending a python
program with a few low-level routines. And there is always the potential
need for transposition operations when going to Fortran because of the
row/column memory ordering discrepancy between C and Fortran.
* Complete roll-your-own solutions using the Numarray C api. This is doable,
but after using blitz arrays, it feels really low-level. Call me lazy, but
after using python I've become fully enamored of writing code which reads as
cleanly as possible.
What I'd like to know is if there is any interest in organizing a bit of a
discussion on this. If an hour can be squeezed in for this in the schedule,
great (and I volunteer to moderate if nobody else wants to). If not, perhaps
some of people involved with these topics (Eric, Travis, Konrad, Todd and
Perry at least, plus whoever else is interested) might be willing to commit to
a lunch/dinner to be spent talking about this.
It would be nice to know something in advance, because one of blitz's current
developers (Julian Cummings) is at Caltech. Perhaps we could ask him to come
in and answer some questions about Blitz's future. Blitz feels like the
lowest-impedance fit to python's semantics, but I'd like to know more about
its long-term feasibility before committing to use it too much. If we set any
kind of schedule for this discussion, I can try to contact Julian.
More information about the Scipy-dev