[Numpy-discussion] Unexpected behavior with numpy array

[Robert Kern]

Damian Eads wrote:
> Here's another question: is there any way to construct a numpy array and 
> specify the buffer address where it should store its values? I ask 
> because I would like to construct numpy arrays that work on buffers that 
> come from mmap.

Can you clarify that a little? By "buffer" do you mean a Python buffer() object? 
By "mmap" do you mean Python's mmap in the standard library?

numpy has a memmap class which subclasses ndarray to wrap a mmapped file. It 
handles the opening and mmapping of the file itself, but it could be subclassed 
to override this behavior to take an already opened mmap object.

In general, if you have a buffer() object, you can make an array from it using 
numpy.frombuffer(). This will be a standard ndarray and won't have the 
conveniences of syncing to disk that the memmap class provides.

If you don't have a buffer() object, but just have a pointer allocated from some 
C code, then you *could* fake an object which exposes the __array_interface__() 
method to describe the memory. The numpy.asarray() constructor will use that to 
make an ndarray object that uses the specified memory. This is advanced stuff 
and difficult to get right because of memory ownership and object lifetime 
issues. If you can modify the C code, it might be easier for you to have numpy 
allocate the memory, then make the C code use that pointer to do its operations.

But look at numpy.memmap first and see if it fits your needs.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco