[SciPy-dev] time series implementation approach
david.huard at gmail.com
Wed Dec 13 08:37:15 CST 2006
For the record:
I use time series to
1. Convert daily data (precip, ET, disharges) to aggregated monthly data.
2. Extract the longest continuous series from a gappy series (sometimes
there are dates but missing values (-99 for instance), and sometimes the
dates themselves are missing).
Wich list: To be able to apply a method on logical chunks of data. For
instance, from a daily series, compute the monthly mean and variance. Or is
that already implemented ?
2006/12/13, Pierre GM <pgmdevlist at gmail.com>:
> Merci beaucoup pour les sources.
> I guess a lot of people in this list have their own implementation of
> Storing a date along data gives indeed the possibility to deal with gapped
> series. However, things becomes quite messy when you have to combine
> series: how do you deal with the gaps? Your approach (for the glance I
> assumes either a step or a linear interpolation, which should work nicely
> your case, but is doubtfully applicable to other cases.
> In fact, you just gave me an idea. In maskedarrays, the mask can be nomask
> (viz, no masked data, surprise), which greatly simplifies most operations.
> Here, we could have a frequency of nofreq, which would indicate that the
> step is not constant: one simple option is then to cast it to the smallest
> reasonable timestep, and set the missing values to "masked". (By
> I mean average: if most of your data is roughly monthly but for a couple
> daily ones, stick to monthly. Unless you really want to go daily. Dammit,
> gonna have to leave this possibility).
> So yeah, we may eventually have to consider varying frequencies. Which
> mean that the shifting array approach will have to be reexamined. Maybe
> if freq is nofreq, then it'll panic, for sure. But if freq is set, then it
> remains a quite viable option.
> OK, now I stop thinking aloud. David, could you tell us more about what
> need ? What kind of data do you have to work with ? Are missing dates
> something you have to deal with on a very regular basis ?
> Scipy-dev mailing list
> Scipy-dev at scipy.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Scipy-dev