[off topic] Re: [Numpy-discussion] numarray speed - PySequence_GetItem

Tim Hochberg tim.hochberg at cox.net
Tue Jun 29 08:15:51 CDT 2004


Todd Miller wrote:

>On Mon, 2004-06-28 at 20:45, Tim Hochberg wrote:
>  
>
>>Todd Miller wrote:
>>
>>    
>>
[SNIP]

>>This reminds me, when profiling bits and pieces of my code I've often 
>>noticed that __del__ chews up a large chunk of time. Is there any 
>>prospect of this being knocked down at all, or is it inherent in the 
>>structure of numarray?
>>    
>>
>
>__del__ is IMHO the elegant way to do numarray's shadowing of
>"misbehaved arrays".  misbehaved arrays are ones which don't meet the
>requirements of a particular C-function, but generally that means
>noncontiguous, byte-swapped, misaligned, or of the wrong type;  it also
>can mean some other sequence type like a list or tuple.  I think using
>the destructor is "necessary" for maintaining Numeric compatibility in C
>because you can generally count on arrays being DECREF'd,  but obviously
>you couldn't count on some new API call being called.  
>  
>
OK, that makes sense. In a general sense at least, I'll have to dig into 
the source to figure out the details.

>__del__ used to be implemented in C as tp_dealloc,  but I was running
>into segfaults which I tracked down to the order in which a new style
>class instance is torn down.  The purpose of __del__ is to copy the
>contents of a well behaved working array (the shadow) back onto the
>original mis-behaved array.  The problem was that, because of the
>numarray class hierarchy, critical pieces of the shadow (the instance
>dictionary) had already been torn down before the tp_dealloc was
>called.  The only way I could think of to fix it was to move the
>destructor farther down in the class hierarchy, i.e. from
>_numarray.tp_dealloc to NumArray.__del__ in Python.
>  
>
It seems that one could stash a reference to the instance dict somewhere 
(in PyArrayObject perhaps) to prevent the instance dict from being torn 
down when the rest of the subclass goes away. It would require another 
field in PyArrayObject , but that's big enough allready that no one 
would notice. I could easily be way off base here though. If I end up 
with too much spare time at some point, I may try to look into this.


>If anyone can think of a way to get rid of __del__, I'm all for it.
>
>  
>
[SNIP]

>>
>>I don't see any easy/obvious ways to speed up wxPySwigInstance_Check, 
>>    
>>
>
>Why no CheckExact, even if it's hand coded?  Maybe the setup is tedious?
>  
>
I don't know although I suspect the setup being tedious might be part of 
it. I also believe that until recently, wxPython used old style classes 
and you couldn't use CheckExact. What I propose below is essentially a 
hand coded version of check exact for wxPoints.

>  
>
>>but I believe that wxPoints now obey the PySequence protocol, so I think 
>>that the whole wxPySwigInstance_Check branch could be removed. To get 
>>that into wxPython you'd probably have to convince Robin that it 
>>wouldn't hurt the speed of list of wxPoints unduly.
>>
>>Wait... If the above doesn't work, I think I do have a way that might 
>>work for speeding the check for a wxPoint. Before the loop starts, get a 
>>pointer to wx.core.Point (the class for wxPoints these days) and call it 
>>wxPoint_Type. Then just use for the check:
>>        o->ob_type == &wxPoint_Type
>>Worth a try anyway.
>>
>>Unfortunately, I don't have any time to try any of this out right now.
>>
>>Chris, are you feeling bored?
>>
>>-tim
>>    
>>
>
>What's the chance of adding direct support for numarray to wxPython? 
>Our PEP reduces the burden on a package to at worst adding 3 include
>files for numarray plus the specialized package code.   With those
>files,  the package can be compiled by users without numarray and also
>run without numarray, but would receive a real boost for people willing
>to install numarray since the sequence protocol could be bypassed.
>  
>
No idea, sorry. I haven't been keeping up with wxPython development lately.

-tim









More information about the Numpy-discussion mailing list