[Numpy-discussion] Subclassing numarray arrays

Colin J. Williams cjw at sympatico.ca
Thu Sep 25 09:15:13 CDT 2003


Todd,

Thanks for the benchmark data.

I intend to post the matrix stuff.  Performance is not a problem for me 
at this stage.

Currently, I'm plodding along, making things work.  Later, I'll post it 
with some rudimentary documentation.

Regarding the "new-classes standard", I can't find much in either the 
Python docs or Alex Martelli's "Python in a Nutshell".  One of the new 
features is a __new__ constructor.

The Python array module doesn't formally have a class and thus it can't 
be sub-classed.  It uses a factory function to create instance objects.

Martelli, in PIAN p74, writes of factory functions being useful when one 
wishes to create instances of different classes depending on some 
condition or for the reuse of an existing instance.

In the Python Cookbook p173, he proposes the use of a factory function 
to get around the limitations of Python 2.1 and earlier.  On page 179, 
he refers to the use of a factory method, again to work around 
limitations before Python 2.2.

My rough and ready definition would include the ability to create an 
instance and initialize
the data in one step.

Perhaps it would be good to post your exchange with Nadev to the 
newsgroup comp.lang.python.  There are many contributors to the group 
who could give a much more complete answer.

Colin W.


Todd Miller wrote:

>On Thu, 2003-09-25 at 08:15, Colin J. Williams wrote:
>  
>
>>Nadav Horesh wrote:
>>[snip]
>>
>>    
>>
>>>As I recall, one of the targets of the numarray project is to be
>>>included as a standard python library --- shouldn't it eventually comply
>>>with the new-classes standard?
>>>      
>>>
>
>Can someone please explain what the "new-classes standard" is? 
>(Obviously meeting it is not one of our current goals)
>
>  
>
>>I hope that more consideration will be given to this as the numarray 
>>package evolves.
>>    
>>
>
>Of course.  We don't need sub-classing internally, however, so it's
>difficult to predict when we'll do better.
>
>  
>
>> From time to time, I am developing a matrix sub-class, it's a bit more 
>>convoluted than I would like it to be.
>>    
>>
>
>Posting it here would be good.  Sending things directly to me tends to
>dumb things down... to me.
> 
>  
>
>>It would also be helpful if a current assessment of the speed of 
>>numarray could be published.  
>>    
>>
>
>You asked... and...  (sorry if this is less than succinct)
>
>The numarray source distribution has a module called bench.py in the
>Examples directory.  Without any analysis,  I've attached the output
>from bench.py on my 1.7 GHz Pentium IV running under i386 linux.  Each
>row shows the timing numbers for numarray and Numeric for a particular
>function call and parameter set;  the important part is the ratio ( < 1
>means numarray is faster.  In general, numarray is slower;  this is due
>to the fact that numarray supports in-place operation on memory mapped
>non-native data formats).
>
>Suspect numbers are... suspect.   Please investigate!  
>
>Caveats:
>
>1) The benchmarks were run on a multi-tasking kernel with no specific
>load.  The benchmarks also vary from platform to platform, processor to
>processor, compiler to compiler, etc.  These are gcc version 3.2
>20020903 (Red Hat Linux 8.0 3.2-7) on a 1.7 GHz P-IV with 384M of RAM.
>
>2) The absolute measured units (microseconds) haven't really been
>validated and shouldn't be taken too seriously.  The relative time
>comparisons should be valid.
>
>3) a,c,d are 1D arrays.  b is a 2D array.
>
>4) the 10**x column shows the logarithmic array size. 
>   2 --> 100 elements, 6 --> 1 million elements
>
>5) The version of bench.py which made these measurements is in CVS, not
>the numarray-0.7 tar-ball.
>
>Wish-list:
>
>1) The right way to eliminate the effects of host load on the
>benchmarks.  
>
>2) A way to create a regression test which includes performance.
>
>  
>
>>I gather that some enhancements arrived with version 0.7.
>>    
>>
>
>I don't recall any performance enhancements.  numarray-0.7 was mostly
>about object array support and bug fixes.  
>
>bench.py results from numarray-0.7 09-25-03:
>
>benchmark                10**x   numarray (usec)    Numeric (usec)  numarray:Numeric
>take(a,c)                    2              1477                25              56.8
>                             3              1592               108              14.7
>                             4              2920               989               2.9
>                             5             15568              9814               1.6
>                             6            136960             96253               1.4
>put(a,c,d)                   2              1673                64              26.1
>                             3              1798               195               9.2
>                             4              3270              1587               2.1
>                             5             16742             15334               1.1
>                             6            147768            151932               1.0
>transpose(b)                 2               154                22               6.8
>                             3               157                24               6.3
>                             4               190                39               4.8
>                             5               262                74               3.5
>                             6               257                67               3.8
>putmask(a,c%2,d)             2              1415               139              10.1
>                             3              1504               271               5.6
>                             4              2439              1735               1.4
>                             5             11036             17337               0.6
>                             6             93927            157848               0.6
>repeat(a,2)                  2              1533               138              11.1
>                             3             10165               579              17.5
>                             4            103064              4701              21.9
>                             5           1028850             58812              17.5
>                             6          10010859            493157              20.3
>choose(a%2,(a,c))            2              1312                73              18.0
>                             3              1389               190               7.3
>                             4              2479              1500               1.7
>                             5             12516             16219               0.8
>                             6            108841            142280               0.8
>ravel(b)                     2               164                38               4.3
>                             3               169                41               4.1
>                             4               196                58               3.4
>                             5               279                93               3.0
>                             6               275                90               3.0
>nonzero(a)                   2              1044               143               7.3
>                             3              1133               285               4.0
>                             4              2108              1929               1.1
>                             5             12002             17974               0.7
>                             6            111598            176598               0.6
>where(a%2,a,c)               2              1273               111              11.4
>                             3              1380               242               5.7
>                             4              2587              1752               1.5
>                             5             13368             16785               0.8
>                             6            117731            164187               0.7
>compress(a%2,a)              2              2489               198              12.6
>                             3              2661               375               7.1
>                             4              4575              2426               1.9
>                             5             22933             22621               1.0
>                             6            200138            219215               0.9
>diagonal(b)                  2              2420             16883               0.1
>                             3              2425             16896               0.1
>                             4              2591             16932               0.2
>                             5              2588             16875               0.2
>                             6              2579             16861               0.2
>trace(b)                     2              2611             16988               0.2
>                             3              2625             16990               0.2
>                             4              2750             17004               0.2
>                             5              2758             17048               0.2
>                             6              2766             16942               0.2
>searchsorted(a,a)            2               506                55               9.2
>                             3               708               490               1.4
>                             4              3054              6507               0.5
>                             5             32101             77762               0.4
>                             6            378341            940912               0.4
>sort(a)                      2               554                85               6.4
>                             3               940               365               2.6
>                             4              5394              3533               1.5
>                             5             55238             36049               1.5
>                             6            607845            420334               1.4
>argsort(a)                   2              1541                90              16.9
>                             3              2061               467               4.4
>                             4              8113              5130               1.6
>                             5             76924             49497               1.6
>                             6            853114            588399               1.4
>argmax(b)                    2           1112081              8692             127.9
>                             3           1110896              9320             119.2
>                             4           1108330              8706             127.3
>                             5           1113449              8720             127.7
>                             6           1112556              8743             127.3
>array(s,typecode='i')        2               229               146               1.6
>                             3               274              1103               0.2
>                             4               764             10900               0.1
>                             5              5450            105774               0.1
>                             6             51500           1253819               0.0
>dot(b,b)                     2           8825828          31402493               0.3
>                             3           8804230          32136602               0.3
>                             4           8748239          31123564               0.3
>                             5           8811900          32910953               0.3
>                             6           8827314          31784649               0.3
>clip(a,50,100)               2              2920               177              16.4
>                             3              3089               370               8.3
>                             4              5138              2471               2.1
>                             5             24340             24737               1.0
>                             6            211277            245540               0.9
>indices((100,100))           2              5253              2696               1.9
>                             3              5230              2634               2.0
>                             4              5290              2687               2.0
>                             5              5383              2732               2.0
>                             6              5558              2733               2.0
>swapaxes(b,0,-1)             2                77                79               1.0
>                             3                77                82               1.0
>                             4                94               106               0.9
>                             5               154               142               1.1
>                             6               146               144               1.0
>concatenate((a,a,a))         2              1021                35              29.1
>                             3              1082                48              22.1
>                             4              1868               431               4.3
>                             5              8843              4945               1.8
>                             6             75181             47495               1.6
>innerproduct(b,b)            2           8719124           8989714               1.0
>                             3           8662431           8840564               1.0
>                             4           8734306           9033604               1.0
>                             5           8694994           8935343               1.0
>                             6           8680599           8869750               1.0
>outerproduct(b[0],b[0])      2             42320             27598               1.5
>                             3             42299             27802               1.5
>                             4             42842             27935               1.5
>                             5             42346             27822               1.5
>                             6             42257             27709               1.5
>resize(a,(2*10**i,))         2              1708               160              10.6
>                             3              1769               171              10.3
>                             4              2648               468               5.6
>                             5              9778              3515               2.8
>                             6             75513             31778               2.4
>a[0]                         2                10                10               1.0
>                             3                10                10               1.0
>                             4                13                12               1.1
>                             5                25                20               1.3
>                             6                25                22               1.1
>b[0]                         2                46                15               3.1
>                             3                48                17               2.8
>                             4                56                18               3.0
>                             5               106                33               3.1
>                             6               100                35               2.9
>a[1000:2000]                 2               303                16              18.0
>                             3               298                15              18.7
>                             4               275                20              13.8
>                             5               354                33              10.4
>                             6               344                34               9.8
>b[400:600,400:600]           2               320                25              12.8
>                             3               317                25              12.2
>                             4               348                30              11.3
>                             5               432                46               9.2
>                             6               418                47               8.7
>arange(10.**i)               2               411                37              11.1
>                             3               448                74               6.0
>                             4               923               570               1.6
>                             5              5630              5592               1.0
>                             6             50735             54222               0.9
>identity(i+2)                2              3147               226              13.9
>                             3              3158               227              13.9
>                             4              3214               258              12.4
>                             5              3311               294              11.3
>                             6              3316               291              11.4
>add(a,c)                     2                76                27               2.7
>                             3                90                40               2.2
>                             4               313               225               1.4
>                             5              2902              2128               1.4
>                             6             28106             19972               1.4
>add(a,c,d)                   2                40                22               1.9
>                             3                46                33               1.4
>                             4               144               132               1.1
>                             5              1128              1165               1.0
>                             6             10316             10827               1.0
>a+c                          2                91                31               2.9
>                             3               105                45               2.4
>                             4               335               231               1.5
>                             5              2945              2130               1.4
>                             6             28102             19744               1.4
>cos(a)                       2                84                53               1.6
>                             3               212               235               0.9
>                             4              1546              2351               0.7
>                             5             15237             23074               0.7
>                             6            151604            227898               0.7
>add.reduce(a)                2                97                31               3.0
>                             3               109                41               2.7
>                             4               207               123               1.7
>                             5              1062               949               1.1
>                             6              8974              8844               1.0
>add.reduce(b,0)              2             32527              9218               3.5
>                             3             31652              9238               3.4
>                             4             31376              9171               3.4
>                             5             31412              9271               3.4
>                             6             30443              9263               3.3
>add.reduce(b,-1)             2              9065              9389               1.0
>                             3              9075              9360               1.0
>                             4              9102              9271               1.0
>                             5              9168              9389               1.0
>                             6              9148              9394               1.0
>add.accumulate(a)            2               100                28               3.5
>                             3               120                41               2.9
>                             4               416               254               1.6
>                             5              3584              2696               1.3
>                             6             34837             25608               1.4
>add.accumulate(b,0)          2             63947             25673               2.5
>                             3             65875             25691               2.6
>                             4             65540             25708               2.5
>                             5             65161             25776               2.5
>                             6             66539             25646               2.6
>add.accumulate(b,1)          2             35048             25756               1.4
>                             3             35022             25738               1.4
>                             4             35048             25688               1.4
>                             5             35092             25761               1.4
>                             6             35110             25686               1.4
>
>
>
>
>-------------------------------------------------------
>This sf.net email is sponsored by:ThinkGeek
>Welcome to geek heaven.
>http://thinkgeek.com/sf
>_______________________________________________
>Numpy-discussion mailing list
>Numpy-discussion at lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>  
>






More information about the Numpy-discussion mailing list