[SciPy-dev] Difficulty creating an example for illustrating memmap.flush()

David Gowers 00ai99@gmail....
Sun Aug 2 18:56:21 CDT 2009


On Mon, Aug 3, 2009 at 7:30 AM, David Goldsmith <d_l_goldsmith@yahoo.com>wrote:

>
> Hi, folks.  So, I'm trying to devise an example to illustrate
> memmap.flush().  It didn't work the way I was intuitively expecting (i.e.,
> it didn't appear to do anything at all) so I went looking for a "ready-made"
> example and found:
>
>
> http://www.slideshare.net/enthought/python-for-scientific-computing-webinar-may-22-2009
>
> which has an example on Slide 25.  However, here's what I get when I try to
> duplicate the example:
>
> Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit
> win32
> >>> import numpy as np
> >>> np.version.version
> '1.3.0rc2'
> >>> q = np.memmap('new_file.dat',mode='w+',shape=(2,5))
> >>> q
> memmap([[0, 0, 0, 0, 0],
>       [0, 0, 0, 0, 0]], dtype=uint8)
> >>> # Print out underlying file contents
> ... # Note: not using iPython, so have to use os.system
> ...
> >>> import os
> >>> os.system('type new_file.dat')
>          0
> >>> # Note: already a little different than Webinar Expl.
> ...
> >>> # Next write ascii value for 'A' (65) into q
> ...
> >>> q[:] = ord('A')
> >>> q
> memmap([[65, 65, 65, 65, 65],
>       [65, 65, 65, 65, 65]], dtype=uint8)
> >>> # Do I need to call flush before file is written to?
> ...
> >>> os.system('type new_file.dat')
> AAAAAAAAAA0
> >>> # No! Does flushing change anything?
> ...
> >>> q.flush()
> >>> os.system('type new_file.dat')
> AAAAAAAAAA0
> >>> # No! Is it because I printed q before checking it on disc?
> ...
> >>> # Start afresh, but don't print memmap before checking it
> ...
> >>> r = np.memmap('new_file2.dat',mode='w+',shape=(2,5))
> >>> os.system('type new_file2.dat') # "reproducibility check"
>          0
> >>> r[:] = ord('A')
> >>> os.system('type new_file2.dat') # Checking file on disc immediately
> AAAAAAAAAA0
> >>> # File is updated without calling flush, indeed without any
> ... # intervening access to the memmap at all!
>
> What gives?  Is this a bug?  If not, can someone please furnish me with an
> example that clearly an explicitly illustrates the function (and necessity)
> of memmap.flush()?
>
Sorry, I can't. I can say that I encountered this when I first discovered
np.memmap and began using it for a lot of things. At one point, I didn't
seem to need to flush; In other cases I did.

I wonder whether it has to do with the underlying i/o buffering (eg. if your
array is smaller than the buffer, all changes are written immediately)

David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/scipy-dev/attachments/20090803/6d56027d/attachment.html 


More information about the Scipy-dev mailing list