[Numpy-discussion] how-to "put" RAM-based numarray into memmap

Todd Miller jmiller at stsci.edu
Mon Nov 24 11:35:08 CST 2003


On Mon, 2003-11-24 at 14:13, Sebastian Haase wrote:
> > Sebastian Haase wrote:
> > >
> > > Hi,
> > > Suppose I have a 500MB-ram Computer and a 300MB ram-only (standard)
> > > numarray.
> > > Now I would like to "save" that onto harddrive (with a small header up
> > > front
> >
> > How about:
> >
> > f = open(filename, 'wb')
> > f.write(MyHeader)
> > A.tofile(f)
> >
> > To read it back in, you need to know where your header ends, by either
> > parsing it or using one of the same size every time, then you can use
> > fromfile() to create an array form it.
> 
> The main reason for my question was just to find out if NUMARRAY supports
> it, and how ?
> Also I have many "bookkeeping" functions already implemented for the
> memmap'd case.
> (That is, I have a class with member methods operting on a member memmapped
> array)
> So if what I descibed is possible I could save myself form duplicating lot's
> of code.
> 
> Essentially I was hoping for the most ellegant solution ;-)
> 

memmap's Memmap class does support an insert() method for adding a new
slice to the end of (or anywhere in) an existing map.  The new slice,
however,  will exist as a block of memory allocated on the heap until
the memmap is saved to disk.  

Thus, two scenarios present themselves: (1) you allocate the new slice
ahead of time and create an array from it, avoiding data duplication (2)
you create an array and later copy it into (a newly inserted slice of)
the memmap, thereby duplicating your data on the heap.

When you close the map,  slices on the heap are written to the map file.

Todd

> Thanks,
> Sebastian
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: SF.net Giveback Program.
> Does SourceForge.net help you be more productive?  Does it
> help you create better code?  SHARE THE LOVE, and help us help
> YOU!  Click Here: http://sourceforge.net/donate/
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller <jmiller at stsci.edu>





More information about the Numpy-discussion mailing list