[SciPy-User] Removing duplicate cols/rows

Sergi Pons Freixes sponsfreixes@gmail....
Mon Dec 19 03:59:16 CST 2011


Hi All,

I'm using a 2D shape array to store pairs of longitudes+latitudes. At
one point, I have to merge two of those 2D arrays, and then remove any
duplicate entry. I've been searching for a function similar to
numpy.unique, but I've had no luck. Any implementation I've been
thinking on looks very "unoptimizied". Is there anything existing
solution, so I do not reinvent the wheel?

To make it clear, I'm looking for:
>>> a = np.array([[1, 1], [2, 3], [1, 1], [5, 4], [2, 3]])
>>> unique_rows(a)
array([[1, 1], [2, 3],[5, 4]])

BTW, I wanted to use just a list of tuples for it, but the lists were
so big that they consumed my 4Gb RAM + 4Gb swap (numpy arrays are more
memory efficient).

Regards,
Sergi


More information about the SciPy-User mailing list