[Numpy-discussion] Reading a big netcdf file

[Christopher Barker]

On 8/3/11 11:09 AM, Ian Stokes-Rees wrote:
> On 8/3/11 12:50 PM, Christopher Barker wrote:
>> As a reference, reading that much data in from a raw file into a numpy
>> array takes 2.57 on my machine (a rather old Mac, but disks haven't
>> gotten much faster).
>
> 2.57 seconds? or minutes?

sorry -- seconds.

>If seconds, does it actually read the whole
> thing into memory in that time, or is there some kind of delayed read
> going on?

I think it reads it all in. However, now that you bring it up, I think 
"timeit" does it a few times, and after the first time, there may well 
be disk cache that speeds things up.

In fact, as I recently wrote the file, there may be disk cache issues 
even on the first read.

I'm no timing expert, but there must be ways to get a clean time.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker@noaa.gov