[Numpy-discussion] Fortran order arrays to and from numpy arrays
Sat Feb 24 19:40:33 CST 2007
Andrew Straw <firstname.lastname@example.org> writes:
> Alexander Schmolck wrote:
> > 2. Despite this overhead, copying around large arrays (e.g. >=1e5 elements) in
> > above way causes notable additional overhead. Whilst I don't think there's
> > a sane way to avoid copying by sharing data between numpy and matlab the
> > copying could likely be done better.
> Alex, what do you think about "hybrid arrays"?
Oh, that's why I said no *sane* way :)
I read about hybrid arrays, but as far as I can tell the only way to use them
to avoid copying stuff around is to create your own hybrid array memory pool
(as you suggested downthread), which seems a highly unattractive pain-for-gain
trade-off, especially given that you *do* have to reorder the data as you go
from numpy to matlab -- unless of course I'm missing something. I have the
impression that just memcpy'ing a single chunk of data isn't that much of a
performance sink, but the reordering copying that mlabwrap currently does
seems rather expensive.
In other words, I think going from matlab to numpy is fine (just memcpy into a
newly created fortran-order numpy array, or more or less equivalently, memcpy
into a C-order array, transpose and reshape), the question appears to be how
to best go from numpy to matlab when the numpy array isn't fortran-contiguous.
I assume that it makes more sense to rely on some numpy functionality than to
use a custom reordering-copy routine, especially if I want to move to ctypes
later. Is there anything better than
1. allocating a matlab array
2. transposing and reshaping the numpy array
3. allocating (or keeping around) a temporary numpy array with data
pointing to the matlab array data
4. using some function (PyArray_CopyInto?) to copy from the transposed,
reshaped numpy array into the temporary numpy array thereby filling the
matlab array with an appropriately reordered copy of the original array
More information about the Numpy-discussion