[Numpy-discussion] Deprecating PyDataMem_RENEW ?
Mon May 5 13:25:48 CDT 2008
On Tue, May 6, 2008 at 1:59 AM, Timothy Hochberg <email@example.com> wrote:
> I don't think you would want to do this in the core of PyArray_FromIter;
> presumably realloc can sometimes reuse the existing pointer and save on
> allocating a new chunk of memory. Since there are lots of allocations in
> fromiter, this could potentially be a big performance hit.
Looking at PyArray_FromIter, there is already a log(n) behavior for
allocation, so I am not sure it would hurt so much: I would guess that
realloc often need to allocate a new block if the size is the double
of the former pointer.
That's how realloc seems to work on FreeBSD, at least:
> (At least I think
> so, realloc has always been kinda voodoo to me).
Once malloc is implemented, I would guess realloc to be simple, no ?
For each block of memory in the malloc chain, just check whereas there
is enough size to reuse the same block and adjacent block if any,
otherwise, free + malloc + copy, non ?
One could use
> PyMemData_NEW/PyMemData_FREE in the final allocation to make sure that the
> data is alligned, we allready do a realloc there to dump any extra space.
I guess that would be the easiest: directly use realloc in the loop,
and use PyDataMem_NEW/FREE at the end if necessary. Using realloc is
no problem if the buffer is always freed in the same function.
> Or, possibly better, one could choose which allocation strategy to use here
> depending on whether the data was alligned or not.
I mentioned step 1, because there would be a step 2 (but maybe only
numpy 2): extending the C api functions to create arrays such as
asking for an explicit alignment is possible.
More information about the Numpy-discussion