[Numpy-discussion] A comparative test
Fernando.Perez at colorado.edu
Thu Oct 6 13:28:36 CDT 2005
Rick White wrote:
> IMHO the numarray developers put their emphasis in the right place
> by focusing on large array performance and improved functionality,
> and all the noise around small array indexing performance was just
> a red herring that convinced some folks not to try numarray because
> they heard it was slow. I hope Travis doesn't devote a lot of
> effort to this optimization right now. I'd be much more interested
> in seeing a large array benchmark.
Except for codes like our PDE solvers, which need to create 10s of millions of
small arrays very, very fast. So no, it's not a red herring . Just
because you don't need something doesn't mean there's no valid need for it.
I am not disputing in any way the value of the many, many improvements made by
numarray, nor the importance of fixing large array performance for many
applications (such as I imagine is the typical workflow of astronomical data
analysis). The fact that Travis tried to incorporate all of numarrays'
improvements into the new library is a recognition of the value of all this work.
But calling the small array performance problem a 'red herring' is inaccurate
and unfair, at best.
 Yes, I could preallocate pools of memory for this, manage them manually,
etc. But then I wouldn't be writing my code in python, would I? The whole
point of using python is to have my cake and eat it too: I want high-level
code that gives me near-hardware max speeds. With carefully tuned (python)
code, judiciously sprinkled minimal amounts of C/C++/Fortran, and a LOT of
thinking about algorithm design, I get that.
More information about the Numpy-discussion