[Numpy-discussion] Making cdecimal.Decimal a native numpy type
Sun Jul 22 08:08:35 CDT 2012
On Sun, Jul 22, 2012 at 8:54 AM, Dr.Leo <email@example.com> wrote:
> I am a seasoned numpy/pandas user mainly interested in financial
> applications. These and other applications would greatly benefit from a
> decimal data type with flexible rounding rules, precision etc.
> Yes, there is cdecimal, the traditional decimal module from the Python
> stdlib rewritten in C,
> - http://www.bytereef.org/mpdecimal/index.html -
> which has become part of the stdlib from Python 3.3.
> However, it appears that cdecimal cannot be meaningfully used with numpy
> (see the benchmark below). Squaring an n=10000 ndarray is 1500 times
> faster with float64 than with a dtype=object ndarray based on
> cdecimal.Decimal, and even simple operations fail in the first place.
> I am not deeply enough into ufuncs etc. to judge if some of these
> problems can be avoided with a few lines of Python code. However, my
> impression is that ultimately we would all benefit from cdecimal.Decimal
> becoming a native numpy type. Put bluntly, cdecimal is a great tool. But
> it is not yet where we most need it.
> The author of cdecimal, Stefan Krah, would probably have a great deal of
> the skillset needed to successfully take such a project forward. He
> happens to have also written the new memoryview implementation of Python
> 3.3. And from recent correspondence I understand he might be willing to
> get involved in an effort to marry numpy and cdecimal.
> The main question is if such project would fit into what core developers
> see as the future of numpy.
> And here is the benchmark:
> In : from numpy import *
> In : from cdecimal import Decimal
> In : r=random.rand(10000)
> In : d=ndarray(10000, dtype=Decimal)
> In : d.dtype
> Out: dtype('object')
> In : r.dtype
> Out: dtype('float64')
> In : for i in range(10000): d[i] = Decimal(r[i])
> In : %timeit r**2
> 100000 loops, best of 3: 14.7 us per loop
> In : %timeit d**2
> 10 loops, best of 3: 21.2 ms per loop
> In : r.var()
> Out: 0.082478142261349557
> In : d.var()
> TypeError Traceback (most recent call last)
> -11-bf09d28e33ab> in <module>()
> ----> 1 d.var()
> NumPy-Discussion mailing list
The numpy-dtypes repository (https://github.com/numpy/numpy-dtypes)
has been created recently as a repository for extension dtypes for
numpy. This would be the natural place for a decimal dtype.
Currently there is a rational and quaternion type, and documentation
on how to implement a new dtype. This project is at an early stage
and moving somewhat slowly, so contributions and input would be quite
More information about the NumPy-Discussion