[Numpy-discussion] Efficient operator overloading

Timothy Hochberg tim.hochberg@ieee....
Wed Apr 18 13:31:06 CDT 2007

On 4/18/07, Sturla Molden <sturla@molden.no> wrote:
> On 4/18/2007 7:33 AM, Anne Archibald wrote:
> >copying. And the scope of improvement would be very limited; an
> expression like A*B+C*D would be much more efficient, probably, if the
> whole expression were evaluated at once for each element (due to
> memory locality and temporary allocation). But it is impossible for
> numpy, sitting inside python as it does, to do that.
> Most numerical array/matrix libraries dependent on operator overloading
> generates temporaries. That is why Fortran is usually perceived as
> superior to C++ for scientific programming. The Fortran compiler knows
> about arrays and can avoid allocating three temporary arrays to evaluate
> and expression like
> y = a * b + c * d
> [SNIP]
> I'd really like to see a Python extension library do this one day. It
> would be very cool and (almost) as efficient as plain Fortran - though
> not quite, we would still get some small temporary objects created. But
> that is a sacrifice I am willing to pay to use Python. We would gain
> some efficacy over Fortran by postponing indefinitely evaluation of
> computations that are not needed, when this is not known at compile
> time.
> Any comments?

I periodically wonder if something like this could be built on top of
numexpr without too much pain. Simply build up the expression on operations
and then evaluate when data is requested. You'd need some sort of Jekyl &
Hyde matrix that spent the first part of it's life unevaluated and then
converted itself to something very much like an array when it evaluated
itself. If might end up being tricky to make sure that everything got
transparently evaluated at the right time though.

I've never had time to try it though.

Sturla Molden
> Ph.D.
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://projects.scipy.org/pipermail/numpy-discussion/attachments/20070418/a93ed4d8/attachment.html 

More information about the Numpy-discussion mailing list