[off topic] Re: [Numpy-discussion] numarray speed - PySequence_GetItem
jmiller at stsci.edu
Mon Jun 28 16:19:06 CDT 2004
On Mon, 2004-06-28 at 17:14, Sebastian Haase wrote:
> On Monday 28 June 2004 12:03 pm, Todd Miller wrote:
> > On Mon, 2004-06-28 at 13:59, Chris Barker wrote:
> > > Sebastian Haase wrote:
> > > > BTW, from the profiling/timing I did you can tell that wxPyPlot
> > > > actually plots 25000 data points in 0.1 secs - so it's _really_ fast
> > > > ...
> > >
> > > Actually, it's probably not that fast, if you are timing on
> > > Linux/wxGTK/X-Windows. X is asyncronous, so what you are timing is how
> > > long it takes your program to tell X what to draw, but it may take
> > > longer than that to actually draw it. However, what you are timing is
> > > all the stuff that is effected by numarray/Numeric.
> > >
> > > I worked on part of the wxPython DC.DrawXXXList stuff, and I really
> > > wanted a Numeric native version, but Robin really didn't want an
> > > additional dependency. We discussed on this list a while back whether
> > > you could compile against Numeric, but let people run without it, and
> > > have it all work unless Someone actually used it. What makes that tricky
> > > is that the functions that test whether a PyObject is a Numeric array
> > > are in Numeric... but it could probably be done if you tried hard enough
> > > (maybe include just that function in wxPython...)
> > numarray-1.0 has two macros for dealing with this: PyArray_Present()
> > and PyArray_isArray(obj). The former (safely) determines that numarray
> > is installed, while the latter determines that numarray is installed and
> > that obj is a NumArray. Both macros serve to guard sections of code
> > which make more extensive use of the numarray C-API to keep them from
> > segfaulting when numarray is not installed. I think this would be easy
> > to do for Numeric as well.
> > One problem is that compiling a "numarray improved" extension requires
> > some portion of the numarray headers. I refactored the numarray
> > includes so that a relatively simple set of 3 files can be used to
> > support the Numeric compatible interface (for numarray). These could
> > either be included in core Python (with a successful PEP) or included in
> > interested packages. This approach adds a small source code burden
> > somewhere, but eliminates the requirement for users to have numarray
> > installed either to run or compile from source.
> > I'll send out the draft PEP later today.
> > Regards,
> > Todd
> My original question was just this: Does anyone know why numarray is maybe 10
> times slower that Numeric with that particular code segment
> (PySequence_GetItem) ?
Well, the short answer is probably: no.
Looking at the numarray sequence protocol benchmarks in
Examples/bench.py, and looking at what wxPython is probably doing
(fetching a 1x2 element array from an Nx2 and then fetching 2 numerical
values from that)... I can't fully nail it down. My benchmarks show
that numarray is 4x slower for fetching the two element array but only
1.1x slower for the two numbers; that makes me expect at most 4x
Noticing the 50k __del__ calls in your profile, I eliminated __del__
(breaking numarray) to see if that was the problem; the ratios changed
to 2.5x slower and 0.9x slower (actually faster) respectively.
The large number of "Check" routines preceding the numarray path (I
count 7 looking at my copy of wxPython) has me a little concerned. I
think those checks are more expensive for numarray because it is a new
style class. I have a hard time imagining a 10x difference overall,
but I think Python does have to traverse the numarray class hierarchy
rather than do a type pointer comparison so they are more expensive.
Is 10x a measured number or a gut feel?
One last thought: because the sequence protocol is being used rather
than raw array access, compiling matplotlib for numarray (or not) is not
More information about the Numpy-discussion