[Numpy-discussion] Efficient reading of binary data

Sebastian Haase haase@msg.ucsf....
Fri Apr 4 05:12:58 CDT 2008


On Fri, Apr 4, 2008 at 11:33 AM, Jarrod Millman <millman@berkeley.edu> wrote:
> On Fri, Apr 4, 2008 at 1:50 AM, Sebastian Haase <haase@msg.ucsf.edu> wrote:
>  >  Hi,
>  >  Accidentally I'm exactly trying to do the same thing right now .....
>  >
>  >  What is the best way of memmapping into a file that is already open !?
>  >
>  >  I have to read some text (header info) off the beginning of the file
>  >  before I know where the data actually starts.
>  >  I could of course get the position at that point ( f.tell() ) close
>  >  the file, and reopen using memmap.
>  >  However this doesn't sound optimal to me ....
>  >
>  >  Any hints ?
>  >  Could numpy's memmap be changed to also accept file-objects, or there
>  >  a "rule" that memmap always has to have access to the entire file ?
>
>  I am getting a little tired, so this may be incorrect.  But I believe
>  Stefan modified memmaps to allow them to be created from file-like
>  object:  http://projects.scipy.org/scipy/numpy/changeset/4856
>
>  Are you running a released version of NumPy or the trunk?  If you
>  aren't using the trunk, could you give it a try?  It would be good to
>  have it tested before the 1.0.5 release.
>
Hi  Jarrod,

Thanks for the reply. Indeed I'm running only N.__version__
'1.0.4.dev4312'

I hope I find time to try the new feature.
To clarify: if the file is already open, and the current position
(f.tell() ) is somewhere in the middle,
would the memmap "see" the file from there ?  Could a "normal" file
access and a concurrent memmap into that same file "step on each
others feet" ?

Thanks,
Sebastian Haase


More information about the Numpy-discussion mailing list