[Numpy-discussion] ANN: Numexpr 1.1, an efficient array evaluator

Andrew Collette h5py@alfven....
Mon Jan 19 18:59:03 CST 2009


Hi Francesc,

Looks like a cool project!  However, I'm not able to achieve the
advertised speed-ups.  I wrote a simple script to try three approaches
to this kind of problem:

1) Native Python code (i.e. will try to do everything at once using temp arrays)
2) Straightforward numexpr evaluation
3) Simple "chunked" evaluation using array.flat views.  (This solves
the memory problem and allows the use of arbitrary Python
expressions).

I've attached the script; here's the output for the expression
"63 + (a*b) + (c**2) + sin(b)"
along with a few combinations of shapes/dtypes.  As expected, using
anything other than "f8" (double) results in a performance penalty.
Surprisingly, it seems that using chunks via array.flat results in
similar performance for f8, and even better performance for other
dtypes.

(100, 100, 100) f4 (average of 10 runs)
Simple:  0.155238199234
Numexpr:  0.278440499306
Chunked:  0.166213512421

(100, 100, 100) f8 (average of 10 runs)
Simple:  0.241649699211
Numexpr:  0.192837905884
Chunked:  0.183888602257

(100, 100, 100, 10) f4 (average of 10 runs)
Simple:  1.56741549969
Numexpr:  3.40679829121
Chunked:  1.83729870319

(100, 100, 100) i4 (average of 10 runs)
Simple:  0.206279683113
Numexpr:  0.210431909561
Chunked:  0.182894086838

FYI, the current tar file (1.1-1) has a glitch related to the VERSION
file; I added to the bug report at google code.

Andrew Collette

On Fri, Jan 16, 2009 at 4:00 AM, Francesc Alted <faltet@pytables.org> wrote:
> ========================
>  Announcing Numexpr 1.1
> ========================
>
> Numexpr is a fast numerical expression evaluator for NumPy.  With it,
> expressions that operate on arrays (like "3*a+4*b") are accelerated
> and use less memory than doing the same calculation in Python.
>
> The expected speed-ups for Numexpr respect to NumPy are between 0.95x
> and 15x, being 3x or 4x typical values.  The strided and unaligned
> case has been optimized too, so if the expresion contains such arrays,
> the speed-up can increase significantly.  Of course, you will need to
> operate with large arrays (typically larger than the cache size of your
> CPU) to see these improvements in performance.
>
> This release is mainly intended to put in sync some of the
> improvements that had the Numexpr version integrated in PyTables.
> So, this standalone version of Numexpr will benefit of the well tested
> PyTables' version that has been in production for more than a year now.
>
> In case you want to know more in detail what has changed in this
> version, have a look at ``RELEASE_NOTES.txt`` in the tarball.
>
>
> Where I can find Numexpr?
> =========================
>
> The project is hosted at Google code in:
>
> http://code.google.com/p/numexpr/
>
>
> Share your experience
> =====================
>
> Let us know of any bugs, suggestions, gripes, kudos, etc. you may
> have.
>
>
> Enjoy!
>
> --
> Francesc Alted
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: exprtest.py
Type: text/x-python
Size: 2076 bytes
Desc: not available
Url : http://projects.scipy.org/pipermail/numpy-discussion/attachments/20090119/ef0d7e50/attachment.py 


More information about the Numpy-discussion mailing list