[Numpy-discussion] 2 GB limit for memmap'ed files

Glen W. Mabey Glen.Mabey at swri.org
Thu Sep 7 14:34:14 CDT 2006


A long time ago, Travis wrote:
>  On a related, but orthogonal note:
> 
>  My understanding is that using memory-mapped files for *very* large 
>  files will require modification to the mmap module in Python --- 
>  something I think we should push.  One part of that process would be 
>  to add the C-struct array interface to the mmap module and the buffer 
>  object -- perhaps this is how we get the array interface into Python 
>  quickly.   Then, if we could make a base-type mmap that did not use 
>  the buffer interface or the sequence interface (similar to the 
>  bigndarray in scipy_core) and therefore by-passed the problems with 
>  Python in those areas, then the current mmap object could inherit from 
>  the base class and provide current functionality while still exposing 
>  the array interface for access to >2GB files on 64-bit systems.
> 
>  Who would like to take up the ball for modifying mmap in Python in 
>  this fashion?
> 
>  -Travis

Did anyone ever "pick up the ball" on this issue?

Glen





More information about the Numpy-discussion mailing list