[Numpy-discussion] iterating over an array
Todd Miller
jmiller at stsci.edu
Fri Jan 14 06:19:22 CST 2005
On Fri, 2005-01-14 at 09:28 +0100, Francesc Altet wrote:
> A Dijous 13 Gener 2005 23:57, Chris Barker va escriure:
> > >>> setup = 'import Numeric as na; a = na.arange(2000);a.shape=
(1000,2)'
> > >>> Timer('for i in range(len(a)): row=a[i]', setup).timeit
(number=1000)
> > 1.97064208984375
> > >>> setup = 'import numarray as na; a = na.arange(2000);a.shape=
(1000,2)'
> > >>> Timer('for i in range(len(a)): row=a[i]', setup).timeit
(number=1000)
> > 27.220904111862183
> >
> > yup! that's it. numarray's indexing is SLOW. So it's not an
iterator
> > issue. Look in the archives of this list for discussion of why
> > numarray's generic indexing is slow. A search for "wxPython
indexing"
> > will probably turn it up.
>
> Well, if you want to really compare generic indexing speed, you can't
mix
> array creation objects in the process, as your example seems to do.
>
> A pure indexing access test would look like:
>
> >>> setup = 'import numarray as na; a = [i*2 for i in range(2000)]'
> >>> Timer('for i in range(len(a)): row=a[i]', setup).timeit
(number=1000)
> 0.48835396766662598 # With Python Lists
> >>> setup = 'import Numeric as na; a = na.arange(2000);a.shape=
(1000*2,)'
> >>> Timer('for i in range(len(a)): row=a[i]', setup).timeit
(number=1000)
> 0.65753912925720215 # With Numeric
> >>> setup = 'import numarray as na; a = na.arange(2000);a.shape=
(1000*2,)'
> >>> Timer('for i in range(len(a)): row=a[i]', setup).timeit
(number=1000)
> 0.89093804359436035 # With numarray
>
> That shows that numarray indexing is slower than Numeric, but not by a
large
> extent (just a 40%). The real problem with numarray (for Ralf's
example) is,
> as is already known, array creation time.
I thought we were done after what Francesc pointed out above, then I
tried this:
from timeit import Timer
setup = 'import numarray as na; a = na.arange(2000,shape=(2000,))'
print "numarray iteration: ", Timer('for i in a: pass', setup).timeit(number=1000)
print "numarray simple indexing, int value:", Timer('for i in range(len(a)): row=a[i]', setup).timeit(number=1000)
setup = 'import Numeric as na; a = na.arange(2000); a.shape=(2000,)'
print "Numeric iteration: ", Timer('for i in a: pass', setup).timeit(number=1000)
print "Numeric simple indexing, int value:", Timer('for i in range(len(a)): row=a[i]', setup).timeit(number=1000)
And got:
numarray iteration: 8.81474900246
numarray simple indexing, int value: 3.61732387543
Numeric iteration: 1.0384759903
Numeric simple indexing, int value: 2.18056321144
This is running on Python-2.3.4 compiled --with-debug using gcc-3.4.2 on
a 1 GHZ Athlon XP and FC3 Linux. Simple indexing returning an int was
66% slower for me, but iteration was 880% slower.
It looks to me like there is room for significant numarray iteration
improvement; I'm not sure how it needs to be done or if Numeric has any
special support.
Regards,
Todd
More information about the Numpy-discussion
mailing list