[Numpy-discussion] iterating over an array

Todd Miller jmiller at stsci.edu
Fri Jan 14 06:19:22 CST 2005


On Fri, 2005-01-14 at 09:28 +0100, Francesc Altet wrote:
> A Dijous 13 Gener 2005 23:57, Chris Barker va escriure:
> >  >>> setup = 'import Numeric as na; a = na.arange(2000);a.shape=
(1000,2)'
> >  >>> Timer('for i in range(len(a)): row=a[i]', setup).timeit
(number=1000)
> > 1.97064208984375
> >  >>> setup = 'import numarray as na; a = na.arange(2000);a.shape=
(1000,2)'
> >  >>> Timer('for i in range(len(a)): row=a[i]', setup).timeit
(number=1000)
> > 27.220904111862183
> > 
> > yup! that's it. numarray's indexing is SLOW. So it's not an
iterator 
> > issue. Look in the archives of this list for discussion of why 
> > numarray's generic indexing is slow. A search for "wxPython
indexing" 
> > will probably turn it up.
> 
> Well, if you want to really compare generic indexing speed, you can't
mix
> array creation objects in the process, as your example seems to do.
> 
> A pure indexing access test would look like:
> 
> >>> setup = 'import numarray as na; a = [i*2 for i in range(2000)]'
> >>> Timer('for i in range(len(a)): row=a[i]', setup).timeit
(number=1000)
> 0.48835396766662598   # With Python Lists
> >>> setup = 'import Numeric as na; a = na.arange(2000);a.shape=
(1000*2,)'
> >>> Timer('for i in range(len(a)): row=a[i]', setup).timeit
(number=1000)
> 0.65753912925720215   # With Numeric
> >>> setup = 'import numarray as na; a = na.arange(2000);a.shape=
(1000*2,)'
> >>> Timer('for i in range(len(a)): row=a[i]', setup).timeit
(number=1000)
> 0.89093804359436035  # With numarray
> 
> That shows that numarray indexing is slower than Numeric, but not by a
large
> extent (just a 40%). The real problem with numarray (for Ralf's
example) is,
> as is already known, array creation time.

I thought we were done after what Francesc pointed out above,  then I
tried this:

from timeit import Timer

setup = 'import numarray as na; a = na.arange(2000,shape=(2000,))'
print "numarray iteration: ", Timer('for i in a: pass', setup).timeit(number=1000)
print "numarray simple indexing, int value:", Timer('for i in range(len(a)): row=a[i]', setup).timeit(number=1000)

setup = 'import Numeric as na; a = na.arange(2000); a.shape=(2000,)'
print "Numeric iteration: ", Timer('for i in a: pass', setup).timeit(number=1000)
print "Numeric simple indexing, int value:", Timer('for i in range(len(a)): row=a[i]', setup).timeit(number=1000)

And got:

numarray iteration:  8.81474900246
numarray simple indexing, int value: 3.61732387543
Numeric iteration:  1.0384759903
Numeric simple indexing, int value: 2.18056321144

This is running on Python-2.3.4 compiled --with-debug using gcc-3.4.2 on
a 1 GHZ Athlon XP and FC3 Linux.  Simple indexing returning an int was
66% slower for me,  but iteration was 880% slower.

It looks to me like there is room for significant numarray iteration
improvement;  I'm not sure how it needs to be done or if Numeric has any
special support.

Regards,
Todd







More information about the Numpy-discussion mailing list