[Numpy-discussion] Is this an indexing bug?

[Travis Oliphant]

Sturla Molden wrote:
> On 6/19/2007 12:19 PM, Sven Schreiber wrote:
>
>   
>> To be more specific, I would expect shape==(4,14).
>>     
>
>
>  >>> h = numpy.zeros((1,4,14))
>  >>> h[0,:,numpy.arange(14)].shape
> (14, 4)
>  >>> h[0,:,:].shape
> (4, 14)
>  >>>
>
>
> h[0,:,numpy.arange(14)] is a case of "sdvanced indexing". You can also 
> see that
>
>  >>> h[0,:,[0,1,2,3,4,5,6,7,8,9,10,11,12,13]].shape
> (14, 4)
>
> Citing from Travis' book, page 83:
>
> "Example 2: Now let X.shape be (10,20,30,40,50) and suppose ind1 and 
> ind2 are broadcastable to the shape (2,3,4). Then X[:,ind1,ind2] has 
> shape (10,2,3,4,40,50) because the (20,30)-shaped subspace from X has 
> been replaced with the (2,3,4) subspace from the indices. However, 
> X[:,ind1,:,ind2,:] has shape (2,3,4,10,30,50) because there is no 
> unambiguous place to drop in the indexing subspace, thus it is tacked-on 
> to the beginning. It is always possible to use .transpose() to move the 
> sups pace anywhere desired. This example cannot be replicated using take."
>
>
> So I think this strange behaviour is actually correct.
>
>   

Yes, Stuart is right.  Now, obviously, "advanced" indexing was not 
designed with this particular case in mind.  But, it is the expected 
outcome given the rule. 

Let's look at the application of the rule to this particular case.  h is 
a (1,4,10) array.   Now, ind1 is 0 and ind2 is [0,1,...,13].  The rules 
for "advanced" indexing apply because ind2 is a list.   Thus, ind1 is 
broadcast to ind2 which means ind1 acts as if it were [0,0,...,0].    
So, the indexing implies an extraction of a (14,)-shaped array from the 
(1,10)-shaped array. 

Now, where should this (14,)-shaped array be attached in the result.  
The rule states that if ind1 and ind2 are next to each other then it 
will replace the (1,10)-shaped portion of the array.  In this case, 
however, they are not right next to each other.  Thus, there is an 
ambiguity regarding where to place the (14,)-shaped array.   The rule 
states that when this kind of ambiguity arises (notice there is no 
special-case checking to see if ind1 or ind2 comes from a scalar), the 
resulting sub-space is placed at the beginning.  Thus, the 
(1,4,10)-shaped array becomes a (14,4)-shaped array on selection using 
h[ind1,:,ind2]

This behavior follows the rule, precisely.   Admittedly it is a bit 
unexpected in this instance, but it does follow a specific rule that can 
be explained once you understand it, and generalizes to all kinds of 
crazy situations where it is more difficult to see what the behavior 
"should" be.

One may complain that

h[ind1][:,ind2] != h[ind1,:,ind2]

but that is generally true when using slices or lists for indexing.


-Travis