[Numpy-discussion] Numpy array from ctypes pointer object?

Albert Strasheim fullung at gmail.com
Wed Jul 12 15:23:32 CDT 2006


Hello all

Various people wrote:
> >  Im curious though:  the several projects recently using ctypes
> > and numpy to wrap libraries (Pygame SDL, OpenGL, svm) must have come
> > across the issue of using a creating a numpy array from a ctypes
> > pointer.  Ill have to look further.
> >
> It depends on whether or not the library creates memory or not.  Not
> every library manages memory (some expect you to pass in memory already
> owned --- this is easy to do already with ctypes and numpy).

I see two main models for sending back data from C land.

One is to have the C function allocate memory and return the pointer to the
caller. This raises the question of who owns the memory. It also leads to
some *very* interesting crashes on Windows when you start freeing memory
allocated in code linked against one runtime in code linked against another
runtime. I think, if you have the option, avoid this kind of thing at all
costs.

The other model has functions that take their normal arguments, and pointers
to buffers where they can store their results. Typically the caller would
also specify the size of the buffer. If the buffer is large enough for the
function to do its work, it uses it. If not, it returns an error code. This
way, one entity is always in charge of the memory.

If you're writing C code that you plan to use with ctypes and NumPy, I think
the second model will lead to more robust code and less time spent debugging
crashes and memory leaks.

libsvm (which I'm wrapping with ctypes) mostly follows the second model,
except when training new models, in which case it returns a pointer to a
newly allocated structure. To deal with this, I keep a pointer to this
allocated memory in a Python object that has the following function:

def __del__(self):
    libsvm.svm_destroy_model(self.model)

By providing this destroy function, libsvm avoids the problem of mixing
allocation and deallocation across runtimes.

Regards,

Albert





More information about the Numpy-discussion mailing list