[Numpy-discussion] Reading a big netcdf file
Wed Aug 3 15:57:19 CDT 2011
This is what I get here:
In : a = np.zeros((21601, 10801), dtype=np.uint16)
In : a.tofile('temp.npa')
In : del a
In : timeit a = np.fromfile('temp.npa', dtype=np.uint16)
1 loops, best of 3: 251 ms per loop
On Wed, Aug 3, 2011 at 10:50 AM, Christopher Barker
> On 8/3/11 9:30 AM, Kiko wrote:
> > I'm trying to read a big netcdf file (445 Mb) using netcdf4-python.
> I've never noticed that netCDF4 was particularly slow for reading
> (writing can be pretty slow some times). How slow is slow?
> > The data are described as:
> please post the results of:
> ncdump -h the_file_name.nc
> So we can see if there is anything odd in the structure (though I don't
> know what it might be)
> Post your code (in the simnd pplest form you can).
> and post your timings and machine type
> Is the file netcdf4 or 3 format? (the python lib will read either)
> As a reference, reading that much data in from a raw file into a numpy
> array takes 2.57 on my machine (a rather old Mac, but disks haven't
> gotten much faster). YOu can test that like this:
> a = np.zeros((21601, 10801), dtype=np.uint16)
> del a
> timeit a = np.fromfile('temp.npa', dtype=np.uint16)
> (using ipython's timeit)
> Christopher Barker, Ph.D.
> Emergency Response Division
> NOAA/NOS/OR&R (206) 526-6959 voice
> 7600 Sand Point Way NE (206) 526-6329 fax
> Seattle, WA 98115 (206) 526-6317 main reception
> NumPy-Discussion mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the NumPy-Discussion