[Numpy-discussion] iterating over an array

[Chris Barker]

Ralf Juengling wrote:
> for data in a:
>    do sth with data
> 
> instead of 
> 
> for i in range(len(a)):
>    data = a[i]   
>    do sth with data

> Do you think 
> it's possible to speed things up by implementing an 
> __iter__ method? 

Frankly, I seriously doubt it would make much difference, the indexing 
operation would have to take a comparable period of time to your:

do sth with data

That is unlikely.

By the way, here is a test with Python lists:

setup = 'import numarray as na; a = [[i*2,i*2+1] for i in range(1000)]'
 >>> Timer('for i in range(len(a)): row=a[i]', setup).timeit(number=1000)
0.37136483192443848

Much faster than the numarray examples( ~ 30 on my machine).

I suspect the real delay here is that each indexing operation has to 
create a new array (even if they do use the same data). Lists just 
return the item.

Also, it's been discussed that numarray's generic indexing is much 
slower than Numeric's, for instance. This has made a huge difference 
when passing arrays into wxPython, for instance. Perhaps that's 
relevant? Here's a test with Numeric vs. numarray:

 >>> setup = 'import Numeric as na; a = na.arange(2000);a.shape=(1000,2)'
 >>> Timer('for i in range(len(a)): row=a[i]', setup).timeit(number=1000)
1.97064208984375
 >>> setup = 'import numarray as na; a = na.arange(2000);a.shape=(1000,2)'
 >>> Timer('for i in range(len(a)): row=a[i]', setup).timeit(number=1000)
27.220904111862183

yup! that's it. numarray's indexing is SLOW. So it's not an iterator 
issue. Look in the archives of this list for discussion of why 
numarray's generic indexing is slow. A search for "wxPython indexing" 
will probably turn it up.

-Chris






-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov