[Numpy-discussion] is it a bug?
Travis E. Oliphant
Thu Mar 12 22:43:13 CDT 2009
>> It's certainly weird, but it's working as designed. Fancy indexing via
>> arrays is a separate subsystem from indexing via slices. Basically,
>> fancy indexing decides the outermost shape of the result (e.g. the
>> leftmost items in the shape tuple). If there are any sliced axes, they
>> are *appended* to the end of that shape tuple.
> x = np.arange(30)
> x.shape = (2,3,5)
> idx = np.array([0,1,3,4])
> e = x[:,:,idx]
> print e.shape
> #---> return (2,3,4) just as me think.
> e = x[0,:,idx]
> print e.shape
> #---> return (4,3).
> e = x[:,0,idx]
> print e.shape
> #---> return (2,4). not (4,2). why these three cases excute so
> # differently?
This is probably best characterized as a wart stemming from a use-case
oversight in the approach created to handle mixing simple indexing and
Basically, you can understand what happens by noting that when when
scalars are used in combination with index arrays, they are treated as
if they were part of an indexing array. In other words 0 is interpreted
as  (or 1 is interpreted as ) when combined with advanced
indexing. This is in part so that scalars will be broadcast to the
shape of any indexing array to correctly handle indexing in other
Then, when advanced indexing is combined with ':' or '...' some special
rules show up in determining the output shape that have to do with
resolving potential ambiguities. It is arguable that the rules for
resolving ambiguities are a bit simplistic and therefore don't handle
some real use-cases very well like the case you show. On the other
hand, simple rules are better even if the rules about combining ':' and
'...' and advanced indexing are not well-known.
So, to be a little more clear about what is going on, define idx2 = 
and then ask what should the shapes of x[idx2, :, idx] and x[:, idx2,
idx] be? Remember that advanced indexing will broadcast idx2 and idx
to the same shape ( in this case (4,) but they could broadcast to any
shape at all). This broadcasted result shape must be somehow combined
with the shape resulting from performing the slice selection.
With x[:, idx2, idx] it is unambiguous to tack the broadcasted shape to
the end of the shape resulting from the slice-selection (i.e.
x[:,0,0].shape). This leads to the (2,4) result.
Now, what about x[idx2, :, idx]? The idx2 and idx are still broadcast
to the same shape which could be any shape (in this particular case it
is (4,)), but the slice-selection is done "in the middle". So, where
should the shape of the slice selection (i.e. x[0,:,0].shape) be placed
in the output shape? At the time this is determined, there is no
notion that idx2 "came from a scalar" and so it could have come from any
array. Therefore, when there is this kind of ambiguity, the code
always places the broadcasted shape at the beginning. Thus, the
result is (4,) + (3,) --> (4.3).
Perhaps it is a bit surprising in this particular case, but it is
working as designed. I admit that this particular asymmetry does
create some cognitive dissonance which leaves something to be desired.
More information about the Numpy-discussion