[Numpy-discussion] load from text files Pull Request Review

[Chris.Barker]

On 8/27/11 11:08 AM, Christopher Jordan-Squire wrote:
> I've submitted a pull request for a new method for loading data from
> text files into a record array/masked record array.

> Click on the link for more info, but the general idea is to create a
> regular expression for what entries should look like and loop over the
> file, updating the regular expression if it's wrong. Once the types
> are determined the file is loaded line by line into a pre-allocated
> numpy array.

nice stuff.

Have you looked at my "accumulator" class, rather than pre-allocating? 
Less the class itself than that ideas behind it. It's easy enough to do, 
and would keep you from having to run through the file twice. The cost 
of memory re-allocation as the array grows is very small.

I've posted the code recently, but let me know if you want it again.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker@noaa.gov