[SciPy-User] timeseries forwardfill

Christopher Clarke cclarke@chrisdev....
Mon Nov 23 07:59:21 CST 2009

Something seems to have gone wrong with my initial reply!!
Anyway, I often encounter the "initial values" use case when I am creating
business day time series out of RDBMS tables using a subset of the
observations in the table.  For example i have a SQL query fragment like
WHERE symbol='SFC' and dateix BETWEEN '2009-01-01' AND '2009-09-01'
Now suppose that 2009-01-02 and 2009-01-05 are missing (trading is sparse on
many of the exchanges i'm dealing with) i am supposed to forward_fill using
the last traded value for SFC which may or may not be 2008-12-31.  Hence i
have a query that find the last traded values and i use these as the
"initial values".

Anyway here is by forward_fill wrapper. Its not very efficient as i'm
copying and forward_fill is copying etc but..
 I'm actually starting to have reservation about the usefulness forward_fill
on 2d as opposed to  the individual the individual series arrays as i am
finding that i've often got to do loads of transformations and checking on
the individuals arrays before i can combine them into  a single ma array for
filling anyway

def forward_fill2(marr,maxgap=None,init_vals=None):
  init_vals a list with  the same no. of elements as marr.shape[1]
    if arr.ndim == 1:
        if init_vals:
            if arr.mask.any() and arr.mask[0]:
                if init_vals:
                    arr[0] = init_vals[0]

        return forward_fill(arr,maxgap)
        n = arr.shape[1]
        if init_vals:
            if len(init_vals) != n:
                raise ValueError, 'Initial Values sequence does no match
number of columns'
            for c in range(len(init_vals)):
                if arr.mask.any() and mask[0,c]:
                        if init_vals[c]:
        arr = ma.hsplit(arr, n)[0]
        return ma.column_stack([forward_fill(np.squeeze(a),maxgap) for a in

On Thu, Nov 19, 2009 at 5:07 AM, Chris Clarke <cclarke@chrisdev.com> wrote:

> Hi
> The "initial value"
> On Nov 18, 2009, at 9:17 PM, Pierre GM wrote:
> On Nov 18, 2009, at 5:18 PM, Chris Clarke wrote:
> Sorry for the later reply.  Yes forward_fill is still there and it
> works!!!
> Good
> But it seemed to have some more capability (initial values, 2d arrays)
> when  it was in the sandbox??
> I may be wrong and mixing up with some other library.
> That does sound familiar, but i don't think it was part of
> scikits.timeseries...
> A patch for 2D would be welcome, I'm not quite sure what you mean by
> initial value, though
> _______________________________________________
> SciPy-User mailing list
> SciPy-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/scipy-user/attachments/20091123/095fb31e/attachment-0001.html 

More information about the SciPy-User mailing list