[Numpy-discussion] A C++ library and the new array interface: the best approach?

Albert Strasheim fullung at gmail.com
Tue Jul 4 03:44:56 CDT 2006


Hello all

On Mon, 03 Jul 2006, Travis Oliphant wrote:

> Fernando Perez wrote:
> > Hi all,
> >
> > but my lack of familiarity with all the details of new type creation got me a 
> > bit lost.  I'm sure the information I need is all there, but right now I don't 
> > really see the forest with all the leaves in my face.  I've also read the 
> > various recent threads on ctypes, as well was the one initiated by Joris:
> >
> > http://article.gmane.org/gmane.comp.python.numeric.general/6296
> >
> > So I'd like to know if SWIG is really the best way out in this particular case 
> > (and any advice on taking advantage of the array interface via SWIG would be 
> > appreciated), or if ctypes or pyrex could be used here.  I'm quite happy using 
> > pyrex in other contexts, but I know it doesn't directly support C++.  However, 
> > since I have access to all the code, perhaps a pure C layer could be used to 
> > bridge the C++/pyrex gap.  Or given the recent praise for ctypes, perhaps that 
> > can be an option?
> 
> I don't have a lot of time to respond right now.  But here are some 
> thoughts.
> 
> I'm not sure I'd use SWIG at this point. 

At least for C code, I would definately agree. If you already know SWIG, 
you might want to try it, but I started without any knowledge of SWIG 
and ctypes when I wanted to wrap a pretty basic library, and after a 
week of puzzling over SWIG, I learned ctypes and wrapped the library in 
a matter of hours.

> I'd probably either
> 
> a) use ctypes to call the C++ library (if it is compiled as a *shared* 
> library) although to be honest I'm not sure if calling a C++ causes any 
> grief. 

If I understand correctly, because each C++ compiler potentially mangles 
C++ identifier names in a different way, there isn't a way for ctypes 
to know what to look for in a shared library.
 
> b) byte-the bullet and use Boost python (which other's who have 
> integrated C++ code with Python seem to really like).  I've never 
> wrapped my mind around Boost python, though, so I'm not much help there.

If there is any way for you to "hide" you C++ code behind an extern "C" 
interface, i.e. write some C-style wrappers on top of your C++ code, 
then you can use ctypes. As luck would have it, this is exactly what the 
libsvm authors did already, which made wrapping libsvm with ctypes 
relatively straightforward.

If you want to achieve things like Python classes deriving from C++ 
classes and vice versa, Boost-Python is what you want to look at. But 
as others have noted, prepare to wait for the compiler when wrapping 
anything of even moderate size.

In Boost-Python world there's Pyste that allows you to generate code. I 
think Pyste uses GCC-XML underneath. I played with Pyste for a day or 
two long ago, but at that stage the code it made differed greatly from 
what I wrote when doing Boost-Python wrapping by hand, so it was a bit 
strange.

In general, I got the feeling that Pyste wasn't flexible enough to wrap 
any kind of C++ library. But maybe you're lucky and your C++ code is 
just right to wrap easily with Pyste.

> c) write a custom Tensor class that either inherits from the ndarray or 
> is a mix-in class with the ndarray as a component.

Don't know much about this option.

In summary (keeping in mind that I'm a bit of a ctypes zealot at 
present), I'd try the "C interface on top of C++ with ctypes" approach, 
and build an ndarray subclass on top these C functions.

Regards,

Albert




More information about the Numpy-discussion mailing list