[Numpy-discussion] Numpy array from ctypes pointer object?

[Travis Oliphant]

Mark Heslep wrote:
> I don't see a clean way to create a numpy array from a ctypes pointer object. Is the __array_interface_ in ctypes the thing thats missing needed to make this happen?  I've followed Albert's  Scipy cookbook on ctypes here
>
> http://www.scipy.org/Cookbook/Ctypes  <-- getting dated now for numpy SVN but the idea is clear
>
> This shows clean ways to
> 1) Create a ctype array object from a numpy array:
>     npptr= nparray.__array_interface__['data'][0]
>     ctarray = (c_double * nparray.size).from_address(npptr)
>
> 2) Create a numpy array from a ctypes array (not a pointer):
>     ctarray = (c_double * 2)()
>     nparray = N.array( ctarray )
>
> But if  I'm starting with say, a POINTER(c_int) and a separate size argument that's returned from a ctypes foreign function or provided by _fields_ element in a ctypes structure, then I'm unclear how to provide the pointer in Python to numpy.  The numpy methods or creating an array in Python as I see it are basically two:
>
> N.array, N.asarray  <-- require a sequence object
> N.fromstring, N.frombuffer  <-- not available unless given c_char_p and even thats wrong if I don't want zero termination.
>   

The problem here is that from Python NumPy has no way to create an 
ndarray from a pointer.   Doing this creates a situtation where it is 
unclear who owns the memory.  It is probably best to wrap the pointer 
into some kind of object exposing the buffer protocol and then pass that 
to frombuffer (or ndarray.__new__). 

When an ndarray is using memory that is not its own, it expects another 
object to be "in charge" of that memory and the ndarray will  point its 
base attribute to it and increment its reference count.   What should 
the object that is "in charge" of the memory be? 

Perhaps a suitable utility function could be created that can work with 
ctypes to create ndarrays from ctypes memory locations and either own or 
disown the data.

This needs to be thought through a bit, however.

-Travis



> The attributes in nparray.__array_interface_ are not writable, so no joy there.
>
> On the C side the PyArray_SimpleNewFromData( ..dimensions, ...data ptr) C API does the job nicely.  Is there a ctypes paradigm for SimpleNew...?
>
> Mark
>
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>