[Numpy-discussion] fast constructor for arrays from byte data (unpickling?)

[Robert Kern]

On Tue, Aug 07, 2001 at 04:33:30PM -0700, Chris Barker wrote:

[snip]

> Thanks, that works, but now I am wondering: what I want is a fast and
> memory efficient way to get the contents of a fjile into a NujmPy array,
> this sure doesn't look any better than:
> 
> a = fromstring(file.read()))

Depends on how large the file is. file.read() creates a temporary string the
size of the file. That string isn't freed until fromstring() finishes and
returns the array object. For a brief time, both the string and the array have
duplicates of the same data taking up space in memory.

I don't know the details of mmap, so it's certainly possible that the only way
that fromstring knows how to access the data is to pull all of it into memory 
first, thus recreating the problem. Alas.

> thanks anyway,
> 
> -Chris

-- 
Robert Kern
kern at caltech.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter