[SciPy-User] improving efficiency of script
Neil Berg
nberg@atmos.ucla....
Tue Nov 2 15:30:58 CDT 2010
Thanks for the suggestions, John. You we're rightly confused about the 'ncfile' in my function...I've cleaned it up. You're (E) suggestion was also right on target- a co-worker of mine actually modified the script that called upon a module for the for loops. Several hundred seconds were shaved off because of this.
On Nov 1, 2010, at 3:38 PM, John wrote:
> A) maybe you could share a data file too? (since you shared such a
> good code example)
>
> B) is the memory usage growing?
>
> C) not sure if it just for the example, but I'm a little confused
> about the 'ncfile'. Is it global? Because you refer to it in the
> function, but in the function declaration you call it in_file??
>
> D) These things give me a headache!
>
> E) I would *highly* recommend using weave or F2Py to create a small
> little module for the code in the loops. This is really simple, and in
> the end a great tool for significant speed ups. Alternatively, I guess
> you should see how the code could be vectorized...
>
> -john
>
> PS: These are mostly *off the cuff* thoughts, hope it's a little helpful
>
>
> On Mon, Nov 1, 2010 at 11:20 PM, Neil Berg <nberg@atmos.ucla.edu> wrote:
>> Hi Scipy community,
>>
>> Attached is a script that interpolates hourly 80-meter wind speeds at each
>> grid point. Interpolation is performed via a cubic spline over the lowest 8
>> vertical wind speeds and then output the 80-m wind speed.
>>
>> Unfortunately, the script is taking about 30 minutes to run for the first
>> month, and then takes longer for each successive month. This is
>> demonstrated below:
>>
>> 195901_3.nc (January)
>> ntime= 744 nlat= 54 nlon= 96
>> 1734.268457 s for computation.
>>
>> 195906_3.nc (June)
>> ntime= 720 nlat= 54 nlon= 96
>> 14578.560365 s for computation.
>>
>> 195912_3.nc (December)
>> ntime= 744 nlat= 54 nlon= 96
>> 33484.765078 s for computation.
>>
>> I don't understand why it takes so much longer for successive months to run;
>> they are all roughly the same time dimension and have exactly the same
>> latitude and longitude dimensions. Do you have any ideas why this is
>> happening? Also, do you have any suggestions on how to shorten the length
>> of time it takes to run the script in the first place? The "tim_idx"
>> for-loop surrounding the interpolation procedure is the dominant factor
>> increasing the run time, but I am stuck on possible ways to shave time off
>> these steps.
>>
>> Thank you in advance,
>>
>> Neil Berg
>> nberg@atmos.ucla.edu
>> ___________________
>> Mac OS X 10.6.4
>> Python/scipy 2.6.1
>> ___________________
>>
>>
>> _______________________________________________
>> SciPy-User mailing list
>> SciPy-User@scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>
>>
>
>
>
> --
> Configuration
> ``````````````````````````
> Plone 2.5.3-final,
> CMF-1.6.4,
> Zope (Zope 2.9.7-final, python 2.4.4, linux2),
> Python 2.6
> PIL 1.1.6
> Mailman 2.1.9
> Postfix 2.4.5
> Procmail v3.22 2001/09/10
> Basemap: 1.0
> Matplotlib: 1.0.0
> _______________________________________________
> SciPy-User mailing list
> SciPy-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
More information about the SciPy-User
mailing list