[Numpy-discussion] Views of memmaps and offset

Olivier Grisel olivier.grisel@ensta....
Sat Sep 22 10:46:16 CDT 2012


There is also a third use case that is problematic on numpy master:

orig = np.memmap('tmp.mmap', dtype=np.float64, shape=100, mode='w+')
orig[:] = np.arange(orig.shape[0]) * -1.0  # negative markers to
detect under / overflows

a = np.memmap('tmp.mmap', dtype=np.float64, shape=50, mode='r+', offset=16)
a[:] = np.arange(50)
b = np.asarray(a[10:])

Now b does not even have a 'filename' attribute anymore. `b.base` is a
python mmap instance but the later is created with a file descriptor.

It would still be possible to use:

from _multiprocessing import address_of_buffer

to find the memory address of the mmap buffer and use than to open new
buffer views on the same memory segment from subprocesses using
`numpy.frombuffer((ctypes.c_byte * n_byte).fromaddress(addr))` but in
case of failure (e.g. the file has been deleted on the HDD) one gets a
segmentation fault instead of a much more userfriendly catchable file
not found exception.


More information about the NumPy-Discussion mailing list