[Numpy-discussion] efficient use of numpy.where() and .any()

Francesc Altet faltet@carabos....
Mon Apr 23 12:57:04 CDT 2007


El dl 23 de 04 del 2007 a les 12:47 -0400, en/na Anne Archibald va
escriure:
> On 23/04/07, Pierre GM <pgmdevlist@gmail.com> wrote:
> 
> > Note that in addition of the bitwise operators, you can use the "logical_"
> > functions. OK, you'll still end up w/ temporaries, but I wonder whether there
> > couldn't be some tricks to bypass that...
> 
> If you're really determined not to make many temps, you can use their
> output arguments and do it all in-place on the first temporary. The
> few times I've rewritten code that way it hasn't made an appreciable
> positive difference in the speed, and it was sometimes significantly
> slower, perhaps because of the increased memory footprint (the temps
> were longer-lived than when I did it by hand). malloc() is really
> really cheap, and garbage collection is also extremely cheap for huge
> arrays.

Or you can use numexpr. Numexpr doesn't need temporaries (provided that
all the arrays in the expresion are contiguous). It only uses a small
buffer for carrying out computations whose size is, when compared with
large matrices, negligible. It is also usally faster than the pure NumPy
approach:

# Pure NumPy
>>> Timer("c=(i>2) | ((f**2>3) & ~(i*f<2))", "import numpy;
i=numpy.arange(10**6); f=numpy.random.rand(10**6)").repeat(3, 10)
[0.55586099624633789, 0.55565214157104492, 0.556427001953125]

# Using Numexpr
>>> Timer("c=numexpr.evaluate('(i>2) | ((f**2>3) & ~(i*f<2))')", "import
tables; import tables.numexpr as numexpr; import numpy;
i=numpy.arange(10**6); f=numpy.random.rand(10**6)").repeat(3, 10)
[0.25569510459899902, 0.25547599792480469, 0.25554895401000977]

I've used here the numexpr instance that comes with PyTables, but you
can use also the one in the scipy's sandbox as it also supports booleans
since some months ago.

Cheers,

-- 
Francesc Altet    |  Be careful about using the following code --
Carabos Coop. V.  |  I've only proven that it works, 
www.carabos.com   |  I haven't tested it. -- Donald Knuth



More information about the Numpy-discussion mailing list