[Numpy-discussion] numpy arrays, data allocation and SIMD alignement

David Cournapeau david@ar.media.kyoto-u.ac...
Wed Aug 8 22:08:45 CDT 2007


Charles R Harris wrote:
> Anne,
>
> On 8/8/07, *Anne Archibald* <peridot.faceted@gmail.com 
> <mailto:peridot.faceted@gmail.com>> wrote:
>
>     On 08/08/2007, Charles R Harris <charlesr.harris@gmail.com
>     <mailto:charlesr.harris@gmail.com>> wrote:
>     >
>     >
>     > On 8/8/07, Anne Archibald <peridot.faceted@gmail.com
>     <mailto:peridot.faceted@gmail.com>> wrote:
>     > > Oh. Well, it's not *terrible*; it gets you an aligned array.
>     But you
>     > > have to allocate the original array as a 1D byte array (to
>     allow for
>     > > arbitrary realignments) and then align it, reshape it, and
>     reinterpret
>     > > it as a new type. Plus you're allocating an extra ndarray
>     structure,
>     > > which will live as long as the new array does; this not only
>     wastes
>     > > even more memory than the portable alignment solutions, it
>     clogs up
>     > > python's garbage collector.
>     >
>     > The ndarray structure doesn't take up much memory, it is the
>     data that is
>     > large and the data is shared between the original array and the
>     slice. Nor
>     > does the data type of the slice need changing, one simply uses
>     the desired
>     > type to begin with, or at least a type of the right size so that
>     a view will
>     > do the job without copies. Nor do I see how the garbage
>     collector will get
>     > clogged up, slices are a common feature of using numpy. The
>     slice method
>     > also has the advantage of being compiler and operating system
>     independent,
>     > there is a reason Intel used that approach.
>
I am not sure to understand which approach to which problem you are 
talking about here ?

IMHO, the discussion is becoming a bit carried away. What I was 
suggesting is
    - being able to check whether a given data buffer is aligned to a 
given alignment (easy)
    - being able to request an aligned data buffer: requires aligned 
memory allocators, and some additions to the API for creating arrays.

This all boils down to the following case: I have a C function which 
requires N bytes aligned data, I want the numpy API to provide this 
capability. I don't understand the discussion on doing it in python: 
first, this means you cannot request a data buffer at the C level, and I 
don't understand the whole discussion on slice, multi dimension and so 
on either: at the C level, different libraries may need different arrays 
formats, and in the case of fftw, all it cares about is the alignment of 
the data pointer. For contiguous, C order arrays, as long as the data 
pointer is aligned, I don't think we need more; are some people familiar 
with the MKL, who could tell whether we need more ?

cheers,

David


More information about the Numpy-discussion mailing list