[Numpy-discussion] numexpr thoughts
David M. Cooke
cookedm at physics.mcmaster.ca
Tue Mar 7 12:36:01 CST 2006
On Tue, Mar 07, 2006 at 12:08:00PM -0700, Tim Hochberg wrote:
> David M. Cooke wrote:
> >On Tue, Mar 07, 2006 at 11:33:45AM -0700, Tim Hochberg wrote:
> >>Tim Hochberg wrote:
> >>>>3. Reduction. I figure this could be done at the end of the program in
> >>>>each loop: sum or multiply the output register. Downcasting the
> >>>>output could be done here too.
> >>I'm still not excited about summing over the whole output buffer though.
> >>That ends up allocating and scanning through a whole extra buffer which
> >>may result in a signifigant speed and memory hit for large arrays. Since
> >>if we're only doing this on the way out, there should be no problem just
> >>allocating a single double (or complex) to do the sum in. On the way
> >>in, this could be set to zero or one based on what the last opcode is
> >>(sum or product). Then the SUM opcode could simply do something like:
> >No, no, we'd just sum over the 128 element output vector (mem), and
> >add the result to cumulative sum. That vector should already be in
> >cache, as the last op would put it there.
> Ah! Ok then. That's what I was thinking of too. For some reason I
> thought you were proposing building the whole result vector then summing it.
> Here's another wrinkle: how do we deal with:
> >>> a = reshape(arange(9), (3,3))
> >>> sum(a)
> array([ 9, 12, 15])
> Just forbid it? For the time being at least?
Well, I think for now, sum(a) = sum(a.ravel()). Doing differently will
mean working with dimensions, which is something else to figure out.
> They didn't seem to have any speed advantage, so I ripped out all the
> compare with constant opcodes amd used COPY_C instead. I'm probably
> going to rip out OP_WHERE_XXC and OP_WHERE_XCX depending on the timings
> there. Should I also kill OP_ADD_C and friends as well while I'm at it?
Don't bother; the version I'm going to check in already has them gone.
|David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/
|cookedm at physics.mcmaster.ca
More information about the Numpy-discussion