[SciPy-user] non contiguous numpy arrays
Travis Oliphant
oliphant at ee.byu.edu
Wed Mar 15 15:37:54 CST 2006
Gary S. Thompson wrote:
>Hi
> I have read the stuff on the numpy interface and understand most of
>if(?) ;-) However I want to interface numpy to an array which is stored
>as a series of data blocks which are not contiguous in memory... Help!
>
>
>
>so for example I hava 3d matrix of 1024*512*128 where data is stored in
>blocks within memory which are 16x16x16 floats an read in from disk on
>demand...
>
>
>
The memory model for numpy arrays requires that you can access the next
element in each dimension by striding a "fixed-number" of bytes.
In other words, to be a single ndarray, element i,j,k of the array, B,
must be at
start_of_data + i*B.stride[0] + j*B.stride[1] + k*B.stride[2]
So, in your case, do these 16x16x16 blocks of floats all exist at
arbitrary memory locations? If so, then I don't see how you can map
that to a single ndarray.
You could, however, write a class that wraps each block as a separate
sub-array and do the indexing calculations yourself to determine which
sub-block the data is stored in.
But, I don't see a way to use a single ndarray to access memory layed
out like that. Perhaps I still don't understand what you are doing.
Are you using a memory map?
-Travis
More information about the SciPy-user
mailing list