[Numpy-discussion] np.any and np.all short-circuiting
Thu Sep 24 16:43:41 CDT 2009
I noticed that python's "any" can be faster than numpy's "any" (and the similarly for "all").
Then I wondered why.
I realized that numpy implements "any" as logical_or.reduce (and "all" as logical_and.reduce).
This means that numpy cannot take advantage of short-circuiting.
Looking at the timings confirmed my suspects.
I think python fetches one element at the time from the array and as soon as any of them is true it returns true.
Instead, numpy goes on until the end of the array even if the very first element is already true.
Looking at the code I think I found a way to fix it.
I don't see a reason why it should not work. It seems to work. But you never know.
I wanted to run the test suite.
I am unable to run the test on the svn version, neither from .../build/lib... nor form a different folder using sys.path.insert(0, '.../build/lib...').
In the first case I get "NameError: name 'numeric' is not defined"
while in the second case zero tests are successfully performed :-)
What is the correct way of running the tests (without installing the development version in the system)?
Is there some expert of the inner numpy core able to tell whether the approach is correct and won't break something?
I opened a ticket for this:
In the following table any(x) is python's version, np.any(x) is numpy's, while *np.any(x)* is mine.
x = np.zeros(100000, dtype=bool)
x[i] = True
x = np.ones(100000, dtype=bool)
x[i] = False
i any(x) np.any(x) *np.any(x)*
// 6.84 ms 831 µs 189 µs
50000 3.41 ms 832 µs 98 µs
10000 683 µs 831 µs 24.7 µs
1000 68.9 µs 859 µs 8.41 µs
100 7.92 µs 888 µs 6.9 µs
10 1.42 µs 832 µs 6.68 µs
0 712 ns 831 µs 6.65 µs
i all(x) np.all(x) *np.all(x)*
// 6.65 ms 676 µs 300 µs
50000 3.32 ms 677 µs 154 µs
10000 666 µs 676 µs 36.4 µs
1000 67.9 µs 686 µs 9.86 µs
100 7.53 µs 677 µs 7.26 µs
10 1.39 µs 676 µs 7.06 µs
0 716 ns 678 µs 6.96 µs
More information about the NumPy-Discussion