[SciPy-User] scipy.io.netcdf question

Phil Morefield philmorefield@yahoo....
Wed Aug 25 13:58:06 CDT 2010


 
Ben-

The .nc files I'm using range from 550 MB to 915 MB. Large, I know. Here is the 
output from ncdump:
 dimensions:
xc = 140 ;
yc = 115 ;
time = UNLIMITED ; // (14600 currently)
variables:
double level ;
level:units = "m" ;
level:long_name = "height" ;
level:axis = "Z" ;
level:positive = "up" ;
level:coordinate_defines = "point" ;
level:actual_range = 2.f, 2.f ;
double yc(yc) ;
yc:units = "m" ;
yc:long_name = "y-coordinate of polar_stereographic projection" ;
yc:standard_name = "projection_y_coordinate" ;
yc:axis = "Y" ;
yc:coordinate_defines = "point" ;
double xc(xc) ;
xc:units = "m" ;
xc:long_name = "x-coordinate of polar_stereographic projection" ;
xc:standard_name = "projection_x_coordinate" ;
xc:axis = "X" ;
xc:coordinate_defines = "point" ;
double time(time) ;
time:long_name = "time" ;
time:calendar = "365_day" ;
time:standard_name = "time" ;
time:axis = "T" ;
time:units = "days since 1968-01-01 00:00:0.0" ;
time:delta_t = "0000-00-00 03:00:00" ;
time:coordinate_defines = "point" ;
time:actual_range = 1095.125f, 2920.f ;
double lon(yc, xc) ;
lon:units = "degrees_east" ;
lon:long_name = "longitude" ;
lon:standard_name = "longitude" ;
lon:axis = "X" ;
double lat(yc, xc) ;
lat:units = "degrees_north" ;
lat:long_name = "latitude" ;
lat:standard_name = "latitude" ;
lat:axis = "Y" ;
char polar_stereographic ;
polar_stereographic:straight_vertical_longitude_from_pole = 263. ;
polar_stereographic:standard_parallel = 60. ;
polar_stereographic:false_easting = 3475000. ;
polar_stereographic:false_northing = 7475000. ;
polar_stereographic:latitude_of_projection_origin = 90. ;
polar_stereographic:grid_mapping_name = "polar_stereographic" ;
polar_stereographic:resolution_at_standard_parallel = 50000. ;
polar_stereographic:earth_radius = 6371000. ;
float tas(time, yc, xc) ;
tas:units = "K" ;
tas:type = "instantaneaous" ;
tas:long_name = "Surface Air Temperature" ;
tas:standard_name = "air_temperature" ;
tas:missing_value = 1.e+020f ;
tas:_FillValue = 1.e+020f ;
tas:actual_range = 204.64909f, 327.28021f ;
tas:add_offset = 0.f ;
tas:scale_factor = 1.f ;
tas:coordinates = "lon lat level" ;
tas:grid_mapping = "polar_stereographic" ;
tas:level_desc = "2 m" ;
tas:grid_desc = "polar_stereographic" ;
// global attributes:
:Conventions = "CF-1.0" ;
:institution = "Ouranos (Ouranos, Montreal, PQ, Canada)" ;
:source = "CRCM(2006):atmosphere:CRCMv4.2.0(50km,29levels):sea 
ice:AMIPII:land:CLASSv2.7" ;
:project_id = "NARCCAP" ;
:realization = "1" ;
:experiment_id = "1968-2000 climate simulation using CGCM3 member #4 (mc_abv 
experiment)" ;
:experiment_id2 = "Ouranos operational simulation afv" ;:history = "Tue Feb 3 
14:36:39 2009: ncrename -a conventions,Conventions tas_CRCM_1971010103.nc\n",
"created 24/04/2004 by Ton Nom (netCDF3.6.1 udunits1.12.4)" ;
:table_id = "Table 2" ;
}
 
And lastly, I was incorrectly about 'nc_type' last time. The value being passed 
to _read_values is only 10. Not sure where I got that big number. Konrad 
Hinsen's ScientificPython module works fine for what I'm doing, but I'd really 
like to find a way to make scipy do the work.
 
Thanks in advance,
 
Phil




________________________________
From: Benjamin Root <ben.root@ou.edu>
To: SciPy Users List <scipy-user@scipy.org>
Sent: Tue, August 24, 2010 5:36:34 PM
Subject: Re: [SciPy-User] scipy.io.netcdf question




On Tue, Aug 24, 2010 at 3:37 PM, Phil Morefield <philmorefield@yahoo.com> wrote:

A question for scipy experts:
>
>I'm trying to read in a very large three dimensional NetCDF file and 
>failing. The code is simple:
>
>import numpy
>from scipy.io import netcdf
>
>ncFile = netcdf.netcdf_file(my_input_file,"r")
>
>Traceback (most recent call last):
>  File "<interactive input>", line 1, in <module>
>  File "C:\Python25\lib\site-packages\scipy\io\netcdf.py", line 43, in __init__
>    self._parse()
>  File "C:\Python25\lib\site-packages\scipy\io\netcdf.py", line 79, in _parse
>    self._var_array()
>  File "C:\Python25\lib\site-packages\scipy\io\netcdf.py", line 135, in 
>_var_array
>    self.variables[name] = self._read_var()
>  File "C:\Python25\lib\site-packages\scipy\io\netcdf.py", line 180, in 
>_read_var
>    attributes = self._att_array()
>  File "C:\Python25\lib\site-packages\scipy\io\netcdf.py", line 119, in 
>_att_array
>    attributes[name] = self._read_values(n, nc_type)
>  File "C:\Python25\lib\site-packages\scipy\io\netcdf.py", line 193, in 
>_read_values
>    count = n * bytes[nc_type-1]
>IndexError: list index out of range
> 
>I went into the scipy code and determined that the variable 'nc_type' is being 
>passed to _read_values with a value of 1333097057, while the list object 'bytes' 
>only holds six items. That explains the "list index out of range". My suspicion 
>is that some identifying information is wrong with my NetCDF file, but I can't 
>figure out where the 'nc_type' variable comes from exactly, so I'm just 
>guessing.
>
>I'd really appreciate some help with this. Any ideas? 
>
>Thanks!
>

Phil,

Can you please do a ncdump -h on the file and post the output here.  That might 
help explain things.  Also, how many megabytes is this file.  It might be 
possible that the code is not properly detecting the newer mode (which contains 
extra data at the beginning of the header) and therefore is reading bad values 
in the wrong spots.

Ben Root

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/scipy-user/attachments/20100825/9ab02b89/attachment.html 


More information about the SciPy-User mailing list