[Numpy-discussion] Need help for implementing a fast clip in numpy (was slow clip)

David Cournapeau david at ar.media.kyoto-u.ac.jp
Fri Jan 12 00:37:50 CST 2007


Charles R Harris wrote:
>
>
> On 1/11/07, *David Cournapeau* <david at ar.media.kyoto-u.ac.jp 
> <mailto:david at ar.media.kyoto-u.ac.jp>> wrote:
>
>     Timothy Hochberg wrote:
>     >
>     >
>     > On 1/11/07, *Christopher Barker* <Chris.Barker at noaa.gov
>     <mailto:Chris.Barker at noaa.gov>
>     > <mailto:Chris.Barker at noaa.gov <mailto:Chris.Barker at noaa.gov>>>
>     wrote:
>     >
>     > [CHOP]
>     >
>     >
>     >     I'd still like to know if anyone knows how to efficiently
>     loop through
>     >     all the elements of a non-contiguous array in C.
>     >
>     >
>     > First, it's not that important if the array is contiguous for this
>     > sort of thing. What you really care about is whether it's
>     > simply-strided (or maybe single-strided would be a better term).
>     > Anyway, the last dimension of the array can be strided without
>     making
>     > things more difficult. All you need to be able to do is to
>     address the
>     > elements of the array as thedata[offset + stride*index].
>     I don't understand why we need to do thedata[offset + stride * index]
>     instead of thedata[index] when the data are aligned ? It looks like I
>     seriously misunderstood the meaning of alignement...
>
>
> I think you are confusing aligned for contiguous. Aligned normally 
> means the data occurs on natural address boundaries for the 
> architecture and item size, say multiples of 4 for 32 bit words on 32 
> bit machines.
I understand that definition of alignment, but I thought it also implied 
that the data buffer has an equal spacing between its elements, and this 
spacing is equal to the size of the type ?


> This contrasts with the case where a word is split across natural 
> boundaries: some architectures support that with decreased 
> performance, others don't. Then there are endianess issues, etc, etc.  
> Anyway, the easiest data to deal with is contiguous data where the 
> data items lie one after the other in (virtual) memory. The next 
> easiest is where the spacing between data elements is fixed,
So does that mean you can have a numpy array of let's say 8 bytes 
double, but that each item's address may be 16 bytes one from each other 
(if we suppose alignment) ?

There are three concepts here:
    1: address alignment: to be sure that data pointer is a multiple of 
the data type (a bit like buffer returned by posix_memalign).
    2: data "packing": for an array with elements of size d bytes, for a 
given element, you get the next one by adding d bytes to the data pointer.
    3: style of indexing, eg for a rank 4, what is the function (a, b, 
c, d) -> i such as array->data[i] = array[a, b, c, d].

    So if I understand you correctly, contiguous is about points 2 and 
3, and not 3 only as I first thought ? I am confused about the relation 
between (in bytes) strides, contiguous and alignment... Because when I 
read the pages 25 to 28 of the numpy ebook about contiguous memory 
layout, we talk only about items indexing and the relationship between 
multi-dimensional numpy indexing and one dimension indexing in the 
actual data buffer (point 3).
    I thought all numpy arrays were packed...

    David


More information about the Numpy-discussion mailing list