[Numpy-discussion] setting decimal accuracy in array operations (scikits.timeseries)

Pauli Virtanen pav@iki...
Wed Mar 3 14:50:15 CST 2010


ke, 2010-03-03 kello 21:09 +0100, Marco Tuckner kirjoitti:
> am using the scikit.timeseries to convert a hourly timeseries to a lower
> frequency unsing the appropriate function [1].
> 
> When I compare the result to the values calculated with a Pivot table in
> Excel there is a difference in the values which reaches quite high
> values in the total sum of all monthly values.
> 
> I found out that the differnec arises from different decimal settings:
> 
> In Python the numbers show:
> 12.88888888
> 
> whereas in Excel I see:
> 12.8888888888888

Typically, the internal precision used in Python and Numpy is
significantly more than what is printed. Most likely, your problem has a
different cause. Are you sure Excel is using a high enough accuracy?

If you want more help, it would be useful to post a self-contained code
that demonstrates the error.

-- 
Pauli Virtanen



More information about the NumPy-Discussion mailing list