[Numpy-discussion] C-API for non-contiguous arrays

David Cournapeau david@ar.media.kyoto-u.ac...
Thu Oct 25 21:01:00 CDT 2007


Oliver Kranz wrote:
> Hi,
>
> I am working on a Python extension module using of the NumPy C-API. The 
> extension module is an interface to an image processing and analysis 
> library written in C++. The C++ functions are exported with 
> boos::python. Currently I am implementing the support of 
> three-dimensional data sets which can consume a huge amount of memory. 
> The 3D data is stored in a numpy.ndarray. This array is passed to C++ 
> functions which do the calculations.
>
> In general, multi-dimensional arrays can be organized in memory in four 
> different ways:
> 1. C order contiguous
> 2. Fortran order contiguous
> 3. C order non-contiguous
> 4. Fortran order non-contiguous
>
> Am I right that the NumPy C-API can only distinguish between three ways 
> the array is organized in memory? These are:
> 1. C order contiguous e.g. with PyArray_ISCONTIGUOUS(arr)
> 2. Fortran order contiguous e.g. with PyArray_ISFORTRAN(arr)
> 3. non-contiguous e.g. with !PyArray_ISCONTIGUOUS(arr) &&
> !PyArray_ISFORTRAN(arr)
>
> So there is no way to find out if a non-contiguous array has C order or 
> Fortran order. The same holds for Python code e.g. by use of the flags.
>
> a.flags.contiguous
> a.flags.fortran
>
> This is very important for me because I just want to avoid to copy every 
> non-contiguous array into a contiguous array. This would be very 
> inefficient. But I can't find an other solution than copying the array.
It is inefficient depending on what you mean by inefficient. 
Memory-wise, copying is obviously inefficient. But speed-wise, copying 
the array into a contiguous array in C order is faster in most if not 
all cases, because of memory access times.

You may want to read the following article from Ulrich Drepper on memory 
and cache:

http://lwn.net/Articles/252125/

cheers,

David


More information about the Numpy-discussion mailing list