[Numpy-discussion] Index Array Performance

Wes McKinney wesmckinn@gmail....
Tue Feb 14 17:38:31 CST 2012


On Tue, Feb 14, 2012 at 4:03 AM, Francesc Alted <francesc@continuum.io> wrote:
> On Feb 14, 2012, at 1:50 AM, Wes McKinney wrote:
> [clip]
>> But:
>>
>> In [40]: timeit hist[i, j]
>> 10000 loops, best of 3: 32 us per loop
>>
>> So that's roughly 7-8x slower than a simple Cython method, so I
>> sincerely hope it could be brought down to the sub 10 microsecond
>> level with a little bit of work.
>
> I vaguely remember this has shown up before.  My hunch is that indexing in NumPy is so powerful, that it has to check for a lot of different values for indices (integers, tuples, lists, arrays…), and that it is all these checks what is taking time.  Your Cython wrapper just assumed that the indices where integers, so this is probably the reason why it is that much faster.
>
> This is not to say that indexing in NumPy could not be accelerated, but it won't be trivial, IMO.
>

Given that __getitem__ and __setitem__ receive a 2-tuple of
1-dimensional integer arrays, should be pretty simple (dare I say
trivial? :) ) to optimize for this use case? The abysmal performance
of of __getitem__ and __setitem__ with 1d integer arrays is pretty
high on my list of annoyances with NumPy (especially when take and put
are so much faster), so you guys may see a pull request from me
whenever I can spare the time to hack on it (assuming you don't beat
me to it)!

> -- Francesc Alted
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion


More information about the NumPy-Discussion mailing list