[SciPy-User] Trend Detection

Pierre GM pgmdevlist@gmail....
Fri Oct 2 14:57:58 CDT 2009

On Oct 2, 2009, at 10:40 AM, Gökhan Sever wrote:

> Hello,
> Recently, I have come across a new paper published with the title:  
> Deterministic versus stochastic trends: Detection and challenges  
> [1]. I am planning to experimentally apply (very preferably, without  
> re-inventing the wheel :) some of the techniques that they adapted  
> for trend detection (parametric and non-parametric ones) on my  
> datasets. (Ahh, I don't know how will it would take for me to fully  
> grasp what this is really means: "including wavelet analysis,  
> heuristic methods and by fitting fractionally integrated  
> autoregressive moving average models.")
> Are any of the mentioned approaches available in SciPy habitat?

* Deseasonalization: if you follow the basic approach of the authors  
(subtract the mean, divide by the std error), you'll find  
scikits.timeseries quite helpful. If you want STL/loess, you can  
easily integrate Cleveland's routines in Scipy w/ f2py (there used to  
be a port somewhere, I know I had one, must still be hidden on a hard  
disk. Contact me).

* Stationarity: I've worked with a method called SiNos (SIgnificant  
NOn-Stationarities)  that test whether variations in mean/variance/ 
lag-1 autocorrelation are significant or not at different time scales.  
Works fine on continuous data, trikcier on 'discrete' ones (for  
example, daily precipitation). It's an adaptation of SiZer  
(Significant Zero-Crossing of the first derivatives), a technique to  
find significant features in a distribution. I have a full Scipy port  
for SiNos, with dosc, not posted yet for different reasons (one is  
that I'm not sure of the license type). More info: http://www3.interscience.wiley.com/journal/119402153/abstract?CRETRY=1&SRETRY=0

* LRD/Hurst coeffs: check the Koutsoyiannis references in the paper,  
they're quite useful. Finding the Hurst coefficient is omething I'd be  
interested in seeing. It's on one of my todo lists, but rather low...

* Trend detection: I have some pieces of code that find the position  
of potential change-points, either with a parametric technique (OLS)  
or a more robust one (derived from Mann-Kendall). You're stuck with  
analyzing the data at the observed time scale and LRD will likely  
throw you off, but that's a good starting point. More info:http:// 

More information about the SciPy-User mailing list