[SciPy-User] improving efficiency of script

Neil Berg nberg@atmos.ucla....
Tue Nov 2 15:30:58 CDT 2010


Thanks for the suggestions, John.  You we're rightly confused about the 'ncfile' in my function...I've cleaned it up.  You're (E) suggestion was also right on target- a co-worker of mine actually modified the script that called upon a module for the for loops.  Several hundred seconds were shaved off because of this.  


On Nov 1, 2010, at 3:38 PM, John wrote:

> A) maybe you could share a data file too? (since you shared such a
> good code example)
> 
> B) is the memory usage growing?
> 
> C) not sure if it just for the example, but I'm a little confused
> about the 'ncfile'. Is it global? Because you refer to it in the
> function, but in the function declaration you call it in_file??
> 
> D) These things give me a headache!
> 
> E) I would *highly* recommend using weave or F2Py to create a small
> little module for the code in the loops. This is really simple, and in
> the end a great tool for significant speed ups. Alternatively, I guess
> you should see how the code could be vectorized...
> 
> -john
> 
> PS: These are mostly *off the cuff* thoughts, hope it's a little helpful
> 
> 
> On Mon, Nov 1, 2010 at 11:20 PM, Neil Berg <nberg@atmos.ucla.edu> wrote:
>> Hi Scipy community,
>> 
>> Attached is a script that interpolates hourly 80-meter wind speeds at each
>> grid point.  Interpolation is performed via a cubic spline over the lowest 8
>> vertical wind speeds and then output the 80-m wind speed.
>> 
>> Unfortunately, the script is taking about 30 minutes to run for the first
>> month, and then takes longer for each successive month.  This is
>> demonstrated below:
>> 
>> 195901_3.nc (January)
>> ntime= 744  nlat= 54  nlon= 96
>> 1734.268457 s for computation.
>> 
>> 195906_3.nc (June)
>> ntime= 720  nlat= 54  nlon= 96
>> 14578.560365 s for computation.
>> 
>> 195912_3.nc (December)
>> ntime= 744  nlat= 54  nlon= 96
>> 33484.765078 s for computation.
>> 
>> I don't understand why it takes so much longer for successive months to run;
>> they are all roughly the same time dimension and have exactly the same
>> latitude and longitude dimensions.  Do you have any ideas why this is
>> happening?  Also, do you have any suggestions on how to shorten the length
>> of time it takes to run the script in the first place?  The "tim_idx"
>> for-loop surrounding the interpolation procedure is the dominant factor
>> increasing the run time, but I am stuck on possible ways to shave time off
>> these steps.
>> 
>> Thank you in advance,
>> 
>> Neil Berg
>> nberg@atmos.ucla.edu
>> ___________________
>> Mac OS X 10.6.4
>> Python/scipy 2.6.1
>> ___________________
>> 
>> 
>> _______________________________________________
>> SciPy-User mailing list
>> SciPy-User@scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>> 
>> 
> 
> 
> 
> -- 
> Configuration
> ``````````````````````````
> Plone 2.5.3-final,
> CMF-1.6.4,
> Zope (Zope 2.9.7-final, python 2.4.4, linux2),
> Python 2.6
> PIL 1.1.6
> Mailman 2.1.9
> Postfix 2.4.5
> Procmail v3.22 2001/09/10
> Basemap: 1.0
> Matplotlib: 1.0.0
> _______________________________________________
> SciPy-User mailing list
> SciPy-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user



More information about the SciPy-User mailing list