[Numpy-discussion] Re: Histograms via indirect index arrays
Piotr Luszczek
luszczek at cs.utk.edu
Fri Mar 17 12:21:01 CST 2006
On Friday 17 March 2006 14:58, Robert Kern wrote:
> Piotr Luszczek wrote:
> > On Friday 17 March 2006 13:29, Travis Oliphant wrote:
> >>how does python interpret
> >>
> >>g[idx] += 1
> >>
> >>How does this get compiled to byte-code?
>
> In [161]: c = compile('g[idx] += 1', '<str>', 'single')
>
> In [162]: import dis
>
> In [163]: dis.dis(c)
> 1 0 LOAD_NAME 0 (g)
> 3 LOAD_NAME 1 (idx)
> 6 DUP_TOPX 2
> 9 BINARY_SUBSCR
> 10 LOAD_CONST 0 (1)
> 13 INPLACE_ADD
> 14 ROT_THREE
> 15 STORE_SUBSCR
> 16 LOAD_CONST 1 (None)
> 19 RETURN_VALUE
This proves my point.
> >>There are two possibilities:
> >>
> >>1) g[idx] creates a new object which then has 1 added to it using
> >>in-place addition.
> >>
> >> This would not produce the desired behavior as g[idx] is a
> >> copy of the data when idx is a
> >> general indexing array as it is in this case. So, you make a
> >>copy of those indices, add 1 to them
> >> and then do what with the resut?
> >>
> >>2) g[idx] += 1 gets converted to g[idx] = g[idx] + 1
> >>
> >> This appears to be effectively what Python actually does.
> >> Notice that there is no way for us to control this behavior
> >> because there is no __inplace_with_indexing_add__ operator to
> >> over-ride.
> >>
> >>There is no such single operation to over-ride for the object. In
> >>other words, I don't see anyay for us to even alter the object to
> >> get the behavior you want from that syntax. We can, of course,
> >> add a function or method to do that, but I we would have to extend
> >> Python to get the behavior you want here.
> >
> > Hardly. At least from what I'm seeing happens on a small example.
> > 'g[idx] += 1' becomes ('g' and 'idx' are generic objects):
> > __getitem__(self, idx)
> > __iadd__(1)
> > __setitem__(result of __iadd__)
> >
> > By design numpy returns views from __getitem__
>
> Only for slices.
>
> In [132]: a = arange(10)
>
> In [133]: idx = [2,2,3]
>
> In [134]: a[idx]
> Out[134]: array([2, 2, 3])
>
> In [135]: b = a[idx]
>
> In [136]: b[-1] = 100
>
> In [137]: a
> Out[137]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
Your example uses lists as indices. This is not interesting.
I'm talking solely about arrays indexing other arrays.
To me it is a special and very important case.
> > In this case, it would be view into 'self' and 'idx' so the
> > __iadd__ would just use the 'idx' directly rather than a copy.
> > Finally, __setitem__ doesn't do anything since 'self' and 'value'
> > will be the same.
>
> No, value is the result of __iadd__ on the temporary array.
>
> 'g[idx] += 1' expands to:
>
> tmp = g.__getitem__(idx)
> val = tmp.__iadd__(1)
> g.__setitem__(idx, val)
You're missing the point. 'tmp' can be of a very specific type
so that 'g.__setitem__' doesn't have to do anything: the 'add 1'
was done by '__iadd__'.
> Given these class definitions:
>
> class A(object):
> def __getitem__(self, idx):
> print 'A.__getitem__(%r)' % idx
> return B()
> def __setitem__(self, idx, value):
> print 'A.__setitem__(%r, %r)' % (idx, value)
>
>
> class B(object):
> def __iadd__(self, x):
> print 'B.__iadd__(%r)' % x
> return self
> def __repr__(self):
> return 'B()'
>
> In [153]: a = A()
>
> In [154]: a[[0, 2, 2, 1]] += 1
> A.__getitem__([0, 2, 2, 1])
> B.__iadd__(1)
> A.__setitem__([0, 2, 2, 1], B())
>
> > Of course, this is just a quick draft. I don't know how it would
> > work in practice and in other cases.
>
> Aye, there's the rub.
Show me a code that breaks.
More information about the Numpy-discussion
mailing list