[SciPy-User] [SciPy-user] numpy and C

David Baddeley david_baddeley@yahoo.com...
Wed Jun 9 18:39:20 CDT 2010


If your description holds, what you're doing is allocating a block of memory (with PyArray_SimpleNew),
then changing the pointer so that it points to your camera buffer, without ever using the memory you allocated. The original memory allocated with PyArray_SimpleNew will get leaked at this point. When Python comes to garbage collect your array, the camera buffer will be dealloced instead of the original block of memory. This sounds all BAD!!! 

I have a feeling that PyArray_SimpleNew also sets the reference count to 1 so there's no need to incref it (although you'd be well advised to check up on this). If this is the case, increfing effectively ensures that the array will never be garbage collected and creates a memory leak.

depending on how the data gets from the camera into the buffer you've got a few options - is it a preallocated buffer which gets constantly refreshed by the camera, or is it a buffer allocated on the fly to hold the results of a command such as camera_get_frame(*buffer).

If it's the first you could either ...

Use PyArray_SimpleNewFromData on your camera buffer, with the caveat that the values in the resulting array will be constantly refreshed from the camera.

or, use memcopy to copy the contents of the buffer to your newly allocated (with PyArray_SimpleNew) array - this way the python array won't change as the camera takes another frame. This also has the advantage that the c code doesn't need to worry about whether python is still using the original buffer before deleting it.

If it's the second the buffer contents won't be changing with time and I'd either use PyArray_SimpleNewFromData, or preferably, as this means you can let python handle the garbage collection for the frame, use PyArray_SimpleNew to allocate an array and pass the data pointer of this array to your camera_get_frame(*buffer) method. If you are stuck with a pre-allocated array and want to keep the python an c memory management as separate as possible, you could also use the memcopy route.

cheers,
David



----- Original Message ----
From: tinauser <tinauser@libero.it>
To: scipy-user@scipy.org
Sent: Thu, 10 June, 2010 2:35:09 AM
Subject: Re: [SciPy-User] [SciPy-user] numpy and C


Dear Charles,

thanks again for the replies.
Why do you say that is difficoult to free memory?
What I do is to allocate the memory(pyincref) before calling the Python
script. The Python script uses then a timer to call a C function to which
the allocated PyArrayObject (created with PyArray SimpleNew) is passed. In
C, the pointer of the PyArray is assigned to a pointer that points to a sort
of data buffer that is filled from a camera. The data buffer is allocated
elsewhere.
When the python GUI is closed, I just decref my PyArrayObject, that I'm
basically using just to pass pointer values. 



Charles R Harris wrote:
> 
> On Wed, Jun 9, 2010 at 7:46 AM, Charles R Harris
> <charlesr.harris@gmail.com>wrote:
> 
>>
>>
>> On Wed, Jun 9, 2010 at 5:38 AM, tinauser <tinauser@libero.it> wrote:
>>
>>>
>>> Dear Charles,
>>> thanks for the reply.
>>> The part of code causing the problem was exactly this
>>>
>>> Pymatout_img->data= cam_frame->data;
>>> where Pymatout is a PyArrayObject and cam_frame is a structure having a
>>> pointer to undefined char data.
>>>
>>> The code works all right if I recast in this way
>>>
>>> Pymatout_img->data= (char*)cam_frame->data;
>>>
>>> I'm not sure if this is allowed;I guessed it works because even if
>>> Pymatout_img->data is always a pointer to char, the PyArrayObject looks
>>> in
>>> ->descr->type_num to see what is the data type.
>>>
>>>
>> Numpy uses char* all over the place and later casts to the needed type,
>> it's the old way of doing void*. So your explicit cast is fine. For some
>> compilers, gcc for example, you also need to use a compiler flag to let
>> the
>> compiler know that you are going to do such things. In gcc the flag is
>> -fno-strict-aliasing but I don't think you need to worry about this in
>> VC.
>>
>> <snip>
>>
>>
> That said, managing the data in this way can be problematic as you need to
> track alignment and worry about freeing of memory. You might want to look
> at
> PyArray SimpleNewFromData.
> 
> Chuck
> 
> _______________________________________________
> SciPy-User mailing list
> SciPy-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
> 
> 

-- 
View this message in context: http://old.nabble.com/numpy-and-C-tp28767579p28831237.html
Sent from the Scipy-User mailing list archive at Nabble.com.

_______________________________________________
SciPy-User mailing list
SciPy-User@scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user



      


More information about the SciPy-User mailing list