[Numpy-discussion] Re: filtering numeric arrays

Gordon Williams g_will at cyberus.ca
Thu Feb 27 12:45:31 CST 2003


Thanks to Tim and Chris.  Just what I was looking for!

I tested both along with some other tries that I had made.


For 10000 points -
G:\GPS\Python\GUI_Test\Filter>speed.py
time for <function listComp at 0x007AE4D8> is 0.022809
time for <function arraykludge at 0x0092AFF0> is 0.060303
time for <function arrayComp at 0x0092AFB0> is 0.055692
time for <function arrayTimH at 0x0092B030> is 0.003652
time for <function arrayChrisB at 0x0092B070> is 0.003561

For 100 points -
G:\GPS\Python\GUI_Test\Filter>speed.py
time for <function listComp at 0x007AE4D8> is 0.000238
time for <function arraykludge at 0x00804BD8> is 0.000784
time for <function arrayComp at 0x00804B98> is 0.000678
time for <function arrayTimH at 0x00804C18> is 0.000376
time for <function arrayChrisB at 0x00804C58> is 0.000153

They scale slightly differently between Tim's and Chris' methods.

Thanks again,

Gordon Williams

Here is the code (since someone will ask):

'''Test the speed of different methods of getting points out of a list'''

import time
import Numeric as n

size= 100
maxNum=size/10.

#data
a= n.array(n.arange(0,maxNum,.05))
a.shape= (size,2)

#list
l= a.tolist()


(xMin,yMin)= (3,2)
(xMax,yMax)= (4,6)

def listComp(seq):
    '''using list comprehension'''
    return [(x,y) for x,y in seq if xMin < x <xMax and yMin< y <yMax]

def arrayComp(seq):
    '''using list comprehension'''
    return n.array([(x,y) for x,y in seq if xMin < x <xMax and yMin< y
<yMax])

def arraykludge(seq):
    '''using numeric stuff'''
    res= n.copy.deepcopy(seq)
    i=0 #index
    for x,y in seq:
        if xMin < x <xMax and yMin< y <yMax:
            res[i]= [x,y]
            i+=1
    return res[:i]

def arrayTimH(seq):
    '''using Tim Hochberg suggestions'''
    cond  = (xMin < seq[:,0]) & (seq[:,0] < xMax) & (yMin < seq[:,1]) &
(seq[:,1] < yMax)
    return n.compress(cond, seq, 0)

def arrayChrisB(seq):
    '''using Chris Barker suggestions'''
    valid = (seq[:,0] > xMin) & (seq[:,0] < xMax) & (seq[:,1] > yMin) &
(seq[:,1] < yMax)
    return n.take(a,n.nonzero(valid))


#Tests
tests= [(listComp,l), (arraykludge,a),(arrayComp,a), (arrayTimH,a),
(arrayChrisB,a)]

for fun,seq in tests:
    t=time.clock()
    apply(fun,(seq,))
    dt= time.clock()-t
    print "time for %s is %f" %(str(fun),dt)

----- Original Message -----
From: "Gordon Williams" <g_will at cyberus.ca>
To: <numpy-discussion at lists.sourceforge.net>
Sent: Thursday, February 27, 2003 2:05 PM
Subject: filtering numeric arrays


> Hi All,
>
> I have an 2D numeric array of x,y points eg [(1,3),(2,4),(5,6)] and I
would
> like to remove all the points in the array that don't meet the min/max
point
> criteria.  I will have several thousand points.  With lists I can do it
like
>
> [(x,y) for x,y in seq if xMin < x <xMax and yMin< y <yMax]
>
> How do I get the same functionality and better speed using numeric.  I
have
> tried a bunch of things using compress and take but I am running up
against
> a brick wall.
>
>
> Any ideas?
>
> Thanks
>
> Gordon Williams
>
>
>
>





More information about the Numpy-discussion mailing list