[Numpy-discussion] Deprecating PyDataMem_RENEW ?
Tue May 6 04:47:15 CDT 2008
Anne Archibald wrote:
> How much does this matter? I mean, what if you simply left all the
> reallocs as-is? The arrays that resulted from reallocs would not
> typically be aligned, but we cannot in any case expect all arrays to
> be aligned.
The problem would be the interaction between the aligned allocator and
realloc. If we do it by "ourselves", then it is no problem, but we
cannot do it if we rely on posix_memalign/memalign. I was not precise
enough when I mentioned portability: I meant that it was not possible to
implement realloc in terms of malloc/free portably, and as such, it is
not possible to implement an aligned realloc with memalign/free. Also, I
have not found any indication that the pointer given by posix_memalign
can be fed to realloc: depending on the implementation, it cannot be
Maybe the solution is to never use posix_memalign...
> As I understand it, a
> portable aligned malloc allocates extra memory, then adds something to
> the pointer to make the memory arena aligned.
Yes, that's how it works in fftw, and the implementation I have for
numpy is derived from fftw code.
> I don't think any numpy function can assume that its input arrays are
> aligned, since users may like to pass, say, mmap()ed hunks of a file,
> over which numpy has no control of the alignment. In this case, then,
> how much of a problem is it if a few stray realloc()s produce
> non-aligned arrays?
There are a few realloc, but they are used in core routines
(PyArray_FromIter, PyArray_Resize). The point is to get aligned arrays
most of the time, and avoiding losing alignement unless it is too
complicated to do otherwise.
More information about the Numpy-discussion