[Numpy-discussion] Access to C/C++, typemaps, pyrex...
Fernando.Perez at colorado.edu
Sun Mar 12 16:34:02 CST 2006
there are recurrent questions on the topic of low-level access to numpy,
wrapping libraries, etc. Many people have home-brewed SWIG typemaps, the
cookbook has some (very nice) pyrex examples, etc. I think that we could use
a bit of unification work on this front, to make this particular issue easier
to deal with for newcomers, and to reduce duplication of effort.
At the last scipy conference, I collected typemaps from a number of people who
had them around and all I could find on the net. I grabbed Scott Ransom's,
Michel Sanner contributed his, Eric Jones gave us some as well, etc. John
Hunter worked on updating Eric's (which had some nice features) to work with
plain C (they were originally C++), and Bill Spotz from Sandia NL agreed to do
the real work of writing some clean, documented examples of typemap use
starting with this codebase.
He gave me this code some time ago, and I shamefully just sat on it. I'm
finally doing something about it, so today I updated this code to work with
numpy instead of Numeric, and all the tests pass.
You can grab it from here with an SVN client:
It is clearly documented, and while I'm sure it can use improvements and
extensions, I think it would be a good starting point to have distributed
/officially/ with numpy. That way others can improve /this/ code, rather than
reimplementing numpy typemaps for the millionth time. D. Beazley (the SWIG
author) has even indicated that he'd be happy to officially ship numpy
typemaps with SWIG if something accepted by the community were sent to him.
After reading the excellent cookbook examples on pyrex usage, which I recently
needed for a project, I decided to complete the code so it could be used 'out
of the box'. This simply meant a bit of organization work, updating the code
to use the 'modern' cimport statement and .pxd files, and writing a proper
setup.py file. The result is here:
I also think that pyrex should be 'officially' supported. In particular, I
think that the c_numpy.pxd file I put there, which is just a copy of that in
mtrand, should be completed and shipped in the same headers directory as the
rest of numpy's .h files. This would make it a LOT easier for newcomers to
use pyrex for writing numpy extensions, since all headers would be complete
and available with all numpy installations.
Finally, if anyone is interested, that repository actually contains in the
examples of how to wrap a more complex library which requires the creation of
C structs from python objects. Getting this right took some time, as not
all the details for this kind of construction are clearly documented anywhere.
I hope it may be useful to others.
I think it would be a good idea to ship the numpy_swig example directory (and
perhaps also numpyx, if somewhat expanded) listed above with the standard
distribution, as well as completing the .pxd file to expose all the numpy C
API for pyrex.
I am not claiming to have done any of the hard work in numpyx or numpy_swig
(though I did spend a lot of time on the wavelets stuff recently :), but I
think we would all benefit from having this kind of 'infrastructure' code in
the core. Once there, improvements can be made over time, with less effort
If we agree on this, it's as simple as dropping those two directories
somewhere on the numpy codebase (sandbox?, doc?). Just having them
'officially' available will be a benefit, I think.
I'd like to thank Bill for the time he put into the swig module, as well as
others who contributed code for this. Hopefully it will benefit everyone.
More information about the Numpy-discussion