[Numpy-discussion] efficient use of numpy.where() and .any()

Francesc Altet faltet@carabos....
Mon Apr 23 12:57:04 CDT 2007

El dl 23 de 04 del 2007 a les 12:47 -0400, en/na Anne Archibald va
> On 23/04/07, Pierre GM <pgmdevlist@gmail.com> wrote:
> > Note that in addition of the bitwise operators, you can use the "logical_"
> > functions. OK, you'll still end up w/ temporaries, but I wonder whether there
> > couldn't be some tricks to bypass that...
> If you're really determined not to make many temps, you can use their
> output arguments and do it all in-place on the first temporary. The
> few times I've rewritten code that way it hasn't made an appreciable
> positive difference in the speed, and it was sometimes significantly
> slower, perhaps because of the increased memory footprint (the temps
> were longer-lived than when I did it by hand). malloc() is really
> really cheap, and garbage collection is also extremely cheap for huge
> arrays.

Or you can use numexpr. Numexpr doesn't need temporaries (provided that
all the arrays in the expresion are contiguous). It only uses a small
buffer for carrying out computations whose size is, when compared with
large matrices, negligible. It is also usally faster than the pure NumPy

# Pure NumPy
>>> Timer("c=(i>2) | ((f**2>3) & ~(i*f<2))", "import numpy;
i=numpy.arange(10**6); f=numpy.random.rand(10**6)").repeat(3, 10)
[0.55586099624633789, 0.55565214157104492, 0.556427001953125]

# Using Numexpr
>>> Timer("c=numexpr.evaluate('(i>2) | ((f**2>3) & ~(i*f<2))')", "import
tables; import tables.numexpr as numexpr; import numpy;
i=numpy.arange(10**6); f=numpy.random.rand(10**6)").repeat(3, 10)
[0.25569510459899902, 0.25547599792480469, 0.25554895401000977]

I've used here the numexpr instance that comes with PyTables, but you
can use also the one in the scipy's sandbox as it also supports booleans
since some months ago.


Francesc Altet    |  Be careful about using the following code --
Carabos Coop. V.  |  I've only proven that it works, 
www.carabos.com   |  I haven't tested it. -- Donald Knuth

More information about the Numpy-discussion mailing list