[Numpy-discussion] Reading a big netcdf file
Wed Aug 3 13:38:08 CDT 2011
On 8/3/11 11:09 AM, Ian Stokes-Rees wrote:
> On 8/3/11 12:50 PM, Christopher Barker wrote:
>> As a reference, reading that much data in from a raw file into a numpy
>> array takes 2.57 on my machine (a rather old Mac, but disks haven't
>> gotten much faster).
> 2.57 seconds? or minutes?
sorry -- seconds.
>If seconds, does it actually read the whole
> thing into memory in that time, or is there some kind of delayed read
> going on?
I think it reads it all in. However, now that you bring it up, I think
"timeit" does it a few times, and after the first time, there may well
be disk cache that speeds things up.
In fact, as I recently wrote the file, there may be disk cache issues
even on the first read.
I'm no timing expert, but there must be ways to get a clean time.
Christopher Barker, Ph.D.
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
More information about the NumPy-Discussion