[SciPy-dev] Difficulty creating an example for illustrating memmap.flush()

David Goldsmith d_l_goldsmith@yahoo....
Sun Aug 2 23:04:11 CDT 2009


OK, thanks, Charles, that explains a lot (which I'll be sure to put in the docstring - don't worry, not all of it, just the "highpoints"), but: if anyone knows a way to actually _prevent_ assignment from writing to the file - and forcing flush to do so right away - so that it may be illustrated with an example, that would still be appreciated (of course, I could "fudge" and call flush and then show the file contents, pretending that it hadn't already happened, but that would be dishonest.)

DG

--- On Sun, 8/2/09, Charles R Harris <charlesr.harris@gmail.com> wrote:

> From: Charles R Harris <charlesr.harris@gmail.com>
> Subject: Re: [SciPy-dev] Difficulty creating an example for illustrating memmap.flush()
> To: "SciPy Developers List" <scipy-dev@scipy.org>
> Date: Sunday, August 2, 2009, 8:48 PM
> 
> 
> On Sun, Aug 2, 2009 at 9:21 PM,
> David Goldsmith <d_l_goldsmith@yahoo.com>
> wrote:
> 
> 
> 
> Hi, David.  Thanks for your (validating) reply. :-)
>  Well, there's something more going on than just the
> size of the mmap or the size of the change to the mmap: I
> experimented with adding a shape (10,) complex128 array to
> shape (10, 10), (10, 10, 10), (10, 10, 10, 10), (10, 10, 10,
> 10, 10) mmaps (and their corresponding "flat"
> mmaps, shape (100,), etc.), both just in the last dimension
> using [9, 9, 9, 9, :], e.g., and everywhere using
> [:,:,:,:,:] - which worked! - and the file on disc was
> updated every time without ever having to call flush, not
> once!  So, hopefully, someone who knows what's going on
> will chime in...
> 
> 
> 
> The data in memmapped files is swapped in and out of memory
> by the operating system, so what happens to be in memory at
> any given time is OS dependent and also changes with demands
> for memory from other processes and such. It's a bunch a
> behind the scenes machinery and not predictable. Even a
> flush can be delayed. My guess is that you need to flush the
> data if you want it to go to disk, which might be the case
> if you have a program that needs to push out data and you
> want to minimise losses from power outages and such.
> Otherwise the data could just sit in memory and never get
> written out until the file is closed.
> 
> 
> Chuck
> 
> 
> 
> 
> -----Inline Attachment Follows-----
> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


More information about the Scipy-dev mailing list