[Numpy-discussion] Need help for implementing a fast clip in numpy (was slow clip)
david at ar.media.kyoto-u.ac.jp
Fri Jan 12 00:37:50 CST 2007
Charles R Harris wrote:
> On 1/11/07, *David Cournapeau* <david at ar.media.kyoto-u.ac.jp
> <mailto:david at ar.media.kyoto-u.ac.jp>> wrote:
> Timothy Hochberg wrote:
> > On 1/11/07, *Christopher Barker* <Chris.Barker at noaa.gov
> <mailto:Chris.Barker at noaa.gov>
> > <mailto:Chris.Barker at noaa.gov <mailto:Chris.Barker at noaa.gov>>>
> > [CHOP]
> > I'd still like to know if anyone knows how to efficiently
> loop through
> > all the elements of a non-contiguous array in C.
> > First, it's not that important if the array is contiguous for this
> > sort of thing. What you really care about is whether it's
> > simply-strided (or maybe single-strided would be a better term).
> > Anyway, the last dimension of the array can be strided without
> > things more difficult. All you need to be able to do is to
> address the
> > elements of the array as thedata[offset + stride*index].
> I don't understand why we need to do thedata[offset + stride * index]
> instead of thedata[index] when the data are aligned ? It looks like I
> seriously misunderstood the meaning of alignement...
> I think you are confusing aligned for contiguous. Aligned normally
> means the data occurs on natural address boundaries for the
> architecture and item size, say multiples of 4 for 32 bit words on 32
> bit machines.
I understand that definition of alignment, but I thought it also implied
that the data buffer has an equal spacing between its elements, and this
spacing is equal to the size of the type ?
> This contrasts with the case where a word is split across natural
> boundaries: some architectures support that with decreased
> performance, others don't. Then there are endianess issues, etc, etc.
> Anyway, the easiest data to deal with is contiguous data where the
> data items lie one after the other in (virtual) memory. The next
> easiest is where the spacing between data elements is fixed,
So does that mean you can have a numpy array of let's say 8 bytes
double, but that each item's address may be 16 bytes one from each other
(if we suppose alignment) ?
There are three concepts here:
1: address alignment: to be sure that data pointer is a multiple of
the data type (a bit like buffer returned by posix_memalign).
2: data "packing": for an array with elements of size d bytes, for a
given element, you get the next one by adding d bytes to the data pointer.
3: style of indexing, eg for a rank 4, what is the function (a, b,
c, d) -> i such as array->data[i] = array[a, b, c, d].
So if I understand you correctly, contiguous is about points 2 and
3, and not 3 only as I first thought ? I am confused about the relation
between (in bytes) strides, contiguous and alignment... Because when I
read the pages 25 to 28 of the numpy ebook about contiguous memory
layout, we talk only about items indexing and the relationship between
multi-dimensional numpy indexing and one dimension indexing in the
actual data buffer (point 3).
I thought all numpy arrays were packed...
More information about the Numpy-discussion