[SciPy-User] timeseries forwardfill
Mon Nov 23 07:59:21 CST 2009
Something seems to have gone wrong with my initial reply!!
Anyway, I often encounter the "initial values" use case when I am creating
business day time series out of RDBMS tables using a subset of the
observations in the table. For example i have a SQL query fragment like
WHERE symbol='SFC' and dateix BETWEEN '2009-01-01' AND '2009-09-01'
Now suppose that 2009-01-02 and 2009-01-05 are missing (trading is sparse on
many of the exchanges i'm dealing with) i am supposed to forward_fill using
the last traded value for SFC which may or may not be 2008-12-31. Hence i
have a query that find the last traded values and i use these as the
Anyway here is by forward_fill wrapper. Its not very efficient as i'm
copying and forward_fill is copying etc but..
I'm actually starting to have reservation about the usefulness forward_fill
on 2d as opposed to the individual the individual series arrays as i am
finding that i've often got to do loads of transformations and checking on
the individuals arrays before i can combine them into a single ma array for
init_vals a list with the same no. of elements as marr.shape
if arr.ndim == 1:
if arr.mask.any() and arr.mask:
arr = init_vals
n = arr.shape
if len(init_vals) != n:
raise ValueError, 'Initial Values sequence does no match
number of columns'
for c in range(len(init_vals)):
if arr.mask.any() and mask[0,c]:
arr = ma.hsplit(arr, n)
return ma.column_stack([forward_fill(np.squeeze(a),maxgap) for a in
On Thu, Nov 19, 2009 at 5:07 AM, Chris Clarke <firstname.lastname@example.org> wrote:
> The "initial value"
> On Nov 18, 2009, at 9:17 PM, Pierre GM wrote:
> On Nov 18, 2009, at 5:18 PM, Chris Clarke wrote:
> Sorry for the later reply. Yes forward_fill is still there and it
> But it seemed to have some more capability (initial values, 2d arrays)
> when it was in the sandbox??
> I may be wrong and mixing up with some other library.
> That does sound familiar, but i don't think it was part of
> A patch for 2D would be welcome, I'm not quite sure what you mean by
> initial value, though
> SciPy-User mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the SciPy-User