[SciPy-user] Very slow comparison of arrays of integers

Brian Granger bgranger at scu.edu
Mon Jul 18 11:33:59 CDT 2005


On Jul 18, 2005, at 1:15 AM, Robert Kern wrote:

> Brian Granger wrote:
>
>> Hello all,
>> I have some code that is using scipy/numeric and one bottleneck  
>> in  the code consists of comparing arrays or lists of integers.   
>> To my  dismay, I am finding that using Python lists is 15-20 times  
>> _faster_  than using numeric array's for this.  The problem is  
>> that it seems  that there is no efficient way to compare arrays of  
>> integers.
>> Here is code that clearly demonstrates this problem:
>> from scipy import *
>> def test_list(n):
>>     a = range(100)
>>     for i in range(n):
>>         r = (a == a)
>> def test_array(n):
>>     a = array(range(100),Int)
>>     for i in range(n):
>>         r = allclose(a,a)
>> The test_list code runs about 20 times as fast as the test_array  
>> code  that uses allclose().
>> Is there any way of comparing to arrays of integers that would be  
>> as  fast or faster than using lists?  Any hints would be greatly   
>> appreciated.
>>
>
> allclose() is for floating point arrays. Use alltrue(x == y) for  
> integer arrays.
>
> Also, use numbers > 100. Small integer objects are cached and  
> object identity is checked first, I believe.
>
> In [28]: tlist = timeit.Timer("a == a", setup="a = range(1000, 2000)")
> In [29]: tarray = timeit.Timer("alltrue(a == a)", setup="from scipy  
> import alltrue,arange; a = arange(1000, 2000)")
>
> In [30]: tarray.repeat(3, 1000)
> Out[30]: [0.097441911697387695, 0.049738168716430664,  
> 0.051641941070556641]
>
> In [31]: tlist.repeat(3, 1000)
> Out[31]: [0.1062159538269043, 0.062600851058959961,  
> 0.063454866409301758]
>

Thanks, alltrue is much faster, especially for large arrays.   
Unfortunately out arrays have only 0's and 1's though, which may be  
slowing us down a bit if their identity is first checked.


> -- 
> Robert Kern
> rkern at ucsd.edu
>
> "In the fields of hell where the grass grows high
>  Are the graves of dreams allowed to die."
>   -- Richard Harter
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.net
> http://www.scipy.net/mailman/listinfo/scipy-user
>



More information about the SciPy-user mailing list