[Numpy-discussion] Fortran order arrays to and from numpy arrays
Alexander Schmolck
a.schmolck@gmx....
Sat Feb 24 19:40:33 CST 2007
Andrew Straw <strawman@astraw.com> writes:
> Alexander Schmolck wrote:
> > 2. Despite this overhead, copying around large arrays (e.g. >=1e5 elements) in
> > above way causes notable additional overhead. Whilst I don't think there's
> > a sane way to avoid copying by sharing data between numpy and matlab the
> > copying could likely be done better.
> >
>
> Alex, what do you think about "hybrid arrays"?
>
> http://www.mail-archive.com/numpy-discussion@lists.sourceforge.net/msg03748.html
Oh, that's why I said no *sane* way :)
I read about hybrid arrays, but as far as I can tell the only way to use them
to avoid copying stuff around is to create your own hybrid array memory pool
(as you suggested downthread), which seems a highly unattractive pain-for-gain
trade-off, especially given that you *do* have to reorder the data as you go
from numpy to matlab -- unless of course I'm missing something. I have the
impression that just memcpy'ing a single chunk of data isn't that much of a
performance sink, but the reordering copying that mlabwrap currently does
seems rather expensive.
In other words, I think going from matlab to numpy is fine (just memcpy into a
newly created fortran-order numpy array, or more or less equivalently, memcpy
into a C-order array, transpose and reshape), the question appears to be how
to best go from numpy to matlab when the numpy array isn't fortran-contiguous.
I assume that it makes more sense to rely on some numpy functionality than to
use a custom reordering-copy routine, especially if I want to move to ctypes
later. Is there anything better than
1. allocating a matlab array
2. transposing and reshaping the numpy array
3. allocating (or keeping around) a temporary numpy array with data
pointing to the matlab array data
4. using some function (PyArray_CopyInto?) to copy from the transposed,
reshaped numpy array into the temporary numpy array thereby filling the
matlab array with an appropriately reordered copy of the original array
?
cheers,
alex
More information about the Numpy-discussion
mailing list