[Numpy-discussion] integer power in scalarmath: how to handle overflow?

Charles R Harris charlesr.harris at gmail.com
Mon Jun 5 18:42:03 CDT 2006

You could use the C approach and use modular arithmetic where the product
simply wraps around. The Python approach would be nice if feasible, but what
are you going to do for integers larger than the largest numpy data type? So
I vote for modular arithetic because numpy is sorta C.

On 6/5/06, David M. Cooke <cookedm at physics.mcmaster.ca> wrote:
> I just ran into the fact that the power function for integer types
> isn't handled in scalarmath yet. I'm going to add it, but I'm wondering
> what people want when power overflows the integer type?
> Taking the concrete example of a = uint8(3), b = uint8(10), then should
> a ** b return
> 1) the maximum integer for the type (255 here)
> 2) 0
> 3) upcast to the largest type that will hold it (but what if it's
>    larger than our largest type? Return a Python long?)
> 4) do the power using "long" like Python does, then downcast it to the
>    type (that would return 169 for the above example)
> 5) something else?
> I'm leaning towards #3; if you do a ** 10, you get the right
> answer (59049 as an int64scalar), because 'a' is upcasted to
> int64scalar, since '10', a Python int, is converted to that type.
> Otherwise, I would choose #1.
> --
> |>|\/|<
> /----------------------------------------------------------------------\
> |David M. Cooke                  http://arbutus.physics.mcmaster.ca/dmc/
> |cookedm at physics.mcmaster.ca
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://projects.scipy.org/pipermail/numpy-discussion/attachments/20060605/3dc1aa7a/attachment-0001.html 

More information about the Numpy-discussion mailing list