[NumPy-Tickets] [NumPy] #1426: subclasses of ndarray have problems with reflected operations in python 3

[NumPy Trac]

#1426: subclasses of ndarray have problems with reflected operations in python 3
------------------------+---------------------------------------------------
 Reporter:  DarrenDale  |       Owner:  somebody
     Type:  defect      |      Status:  new     
 Priority:  high        |   Milestone:  2.0.0   
Component:  numpy.core  |     Version:  devel   
 Keywords:  python-3    |  
------------------------+---------------------------------------------------
 Quoting a discussion on the mailing list, where Pauli V. explained the
 reasons for the reported behavior:

 >> Now that the trunk has some support for python3, I am working on making
 Quantities work with python3 as well. I'm running into some problems
 related to subclassing ndarray that can be illustrated with a simple
 script, reproduced below. It looks like there is a problem with the
 reflected operations, I see problems with __rmul__ and __radd__, but not
 with __mul__ and __add__:
 >
 >Thanks for testing. I wish the test suite was more complete (hint! hint!
 :)
 >
 >Yes, Python 3 introduced some semantic changes in how subclasses of
 builtin classes (= written in C) inherit the __r*__ operations.
 >
 >Below I'll try to explain what is going on. We probably need to change
 some things to make things work better on Py3, within the bounds we are
 able to.
 >
 >Suggestions are welcome. The most obvious one could be to explicitly
 implement __rmul__ etc. on Python 3.

 {{{
 import numpy as np

 class A(np.ndarray):
     def __new__(cls, *args, **kwargs):
         return np.ndarray.__new__(cls, *args, **kwargs)

 class B(A):
     def __mul__(self, other):
         return self.view(A).__mul__(other)
     def __rmul__(self, other):
         return self.view(A).__rmul__(other)
     def __add__(self, other):
         return self.view(A).__add__(other)
     def __radd__(self, other):
         return self.view(A).__radd__(other)

 a = A((10,))
 b = B((10,))

 print('A __rmul__:')
 print(a.__rmul__(2))
 # yields NotImplemented
 print(a.view(np.ndarray).__rmul__(2))
 # yields NotImplemented
 }}}

 >Correct. ndarray does not implement __rmul__, but relies on an automatic
 wrapper generated by Python.
 >
 >The automatic wrapper (wrap_binaryfunc_r) does the following:
 >
 >1. Is `type(other)` a subclass of `type(self)`? If yes, call __mul__ with
 swapped arguments.
 >2. If not, bail out with NotImplemented.
 >
 >So it bails out.
 >
 >Previously, the ndarray type had a flag that made Python to skip the
 subclass check. That does not exist any more on Python 3, and is the root
 of this issue.

 {{{
 print(2*a)
 # ok !!??
 }}}

 >Here, Python checks
 >
 >1. Does nb_multiply from the left op succeed? Nope, since floats don't
 know how to multiply ndarrays.
 >
 >2. Does nb_multiply from the right op succeed? Here the execution passes
 *directly* to array_multiply, completely skipping the __rmul__ wrapper.
 >
 >Note also that in the C-level number protocol there is only a single
 multiplication function for both left and right multiplication.
 >

 {{{
 print('B __rmul__:')
 print(b.__rmul__(2))
 # yields NotImplemented
 print(b.view(A).__rmul__(2))
 # yields NotImplemented
 print(b.view(np.ndarray).__rmul__(2))
 # yields NotImplemented
 print(2*b)
 # yields: TypeError: unsupported operand type(s) for *: 'int' and 'B'
 }}}

 >But here, the subclass calls the wrapper ndarray.__rmul__, which wants to
 be careful with types, and hence fails.
 >
 >Yes, probably explicitly defining __rmul__ for ndarray could be the
 >right solution. Please file a bug report on this.

-- 
Ticket URL: <http://projects.scipy.org/numpy/ticket/1426>
NumPy <http://projects.scipy.org/numpy>
My example project