[Numpy-discussion] Accumulate values that are below threshold

[Bevan Jenkins]

Stéfan van der Walt <stefan <at> sun.ac.za> writes:

> 
> Hi Bevan
> 
> Since the number of output elements are unknown, I don't think you can
> implement this efficiently using arrays.  If your dataset isn't too
> large, a for-loop should do the trick.  Otherwise, you may have to run
> your code through Cython, which optimises for-loops around Python
> lists.
> 
> thresh = 1.0
> carry = 0
> output = []
> for idx, val in data:
>     carry += val
>     if (carry - thresh) >= -1e-15:
>         output.append((idx, carry))
>         carry = 0
> 
> The comparison line above, "(carry - thresh0 >= -1e-15", may look
> strange -- it basically just does "carry >= thresh".  For some reason
> I don't quite understand, when accumulating floats, it sometimes
> happens that "1.0 != 1.0", so I use 1e-15 as protection.
> 
> Regards
> Stéfan
> 
> 2009/1/8 Bevan Jenkins <bevan07 <at> gmail.com>:

Stefan,

Thanks for your solution, it does exactly what i require.  At the moment I am 
just running through proof of concept stuff so the dataset is small.  If I 
start to run into issues with the real data, then that might be the push I 
need to look into Cython.

Bevan