[Numpy-discussion] Subclassing numarray arrays

Todd Miller jmiller at stsci.edu
Thu Sep 25 07:26:11 CDT 2003


On Thu, 2003-09-25 at 08:15, Colin J. Williams wrote:
> Nadav Horesh wrote:
> [snip]
> 
> >As I recall, one of the targets of the numarray project is to be
> >included as a standard python library --- shouldn't it eventually comply
> >with the new-classes standard?

Can someone please explain what the "new-classes standard" is? 
(Obviously meeting it is not one of our current goals)

> I hope that more consideration will be given to this as the numarray 
> package evolves.

Of course.  We don't need sub-classing internally, however, so it's
difficult to predict when we'll do better.

>  From time to time, I am developing a matrix sub-class, it's a bit more 
> convoluted than I would like it to be.

Posting it here would be good.  Sending things directly to me tends to
dumb things down... to me.
 
> It would also be helpful if a current assessment of the speed of 
> numarray could be published.  

You asked... and...  (sorry if this is less than succinct)

The numarray source distribution has a module called bench.py in the
Examples directory.  Without any analysis,  I've attached the output
from bench.py on my 1.7 GHz Pentium IV running under i386 linux.  Each
row shows the timing numbers for numarray and Numeric for a particular
function call and parameter set;  the important part is the ratio ( < 1
means numarray is faster.  In general, numarray is slower;  this is due
to the fact that numarray supports in-place operation on memory mapped
non-native data formats).

Suspect numbers are... suspect.   Please investigate!  

Caveats:

1) The benchmarks were run on a multi-tasking kernel with no specific
load.  The benchmarks also vary from platform to platform, processor to
processor, compiler to compiler, etc.  These are gcc version 3.2
20020903 (Red Hat Linux 8.0 3.2-7) on a 1.7 GHz P-IV with 384M of RAM.

2) The absolute measured units (microseconds) haven't really been
validated and shouldn't be taken too seriously.  The relative time
comparisons should be valid.

3) a,c,d are 1D arrays.  b is a 2D array.

4) the 10**x column shows the logarithmic array size. 
   2 --> 100 elements, 6 --> 1 million elements

5) The version of bench.py which made these measurements is in CVS, not
the numarray-0.7 tar-ball.

Wish-list:

1) The right way to eliminate the effects of host load on the
benchmarks.  

2) A way to create a regression test which includes performance.

> I gather that some enhancements arrived with version 0.7.

I don't recall any performance enhancements.  numarray-0.7 was mostly
about object array support and bug fixes.  

bench.py results from numarray-0.7 09-25-03:

benchmark                10**x   numarray (usec)    Numeric (usec)  numarray:Numeric
take(a,c)                    2              1477                25              56.8
                             3              1592               108              14.7
                             4              2920               989               2.9
                             5             15568              9814               1.6
                             6            136960             96253               1.4
put(a,c,d)                   2              1673                64              26.1
                             3              1798               195               9.2
                             4              3270              1587               2.1
                             5             16742             15334               1.1
                             6            147768            151932               1.0
transpose(b)                 2               154                22               6.8
                             3               157                24               6.3
                             4               190                39               4.8
                             5               262                74               3.5
                             6               257                67               3.8
putmask(a,c%2,d)             2              1415               139              10.1
                             3              1504               271               5.6
                             4              2439              1735               1.4
                             5             11036             17337               0.6
                             6             93927            157848               0.6
repeat(a,2)                  2              1533               138              11.1
                             3             10165               579              17.5
                             4            103064              4701              21.9
                             5           1028850             58812              17.5
                             6          10010859            493157              20.3
choose(a%2,(a,c))            2              1312                73              18.0
                             3              1389               190               7.3
                             4              2479              1500               1.7
                             5             12516             16219               0.8
                             6            108841            142280               0.8
ravel(b)                     2               164                38               4.3
                             3               169                41               4.1
                             4               196                58               3.4
                             5               279                93               3.0
                             6               275                90               3.0
nonzero(a)                   2              1044               143               7.3
                             3              1133               285               4.0
                             4              2108              1929               1.1
                             5             12002             17974               0.7
                             6            111598            176598               0.6
where(a%2,a,c)               2              1273               111              11.4
                             3              1380               242               5.7
                             4              2587              1752               1.5
                             5             13368             16785               0.8
                             6            117731            164187               0.7
compress(a%2,a)              2              2489               198              12.6
                             3              2661               375               7.1
                             4              4575              2426               1.9
                             5             22933             22621               1.0
                             6            200138            219215               0.9
diagonal(b)                  2              2420             16883               0.1
                             3              2425             16896               0.1
                             4              2591             16932               0.2
                             5              2588             16875               0.2
                             6              2579             16861               0.2
trace(b)                     2              2611             16988               0.2
                             3              2625             16990               0.2
                             4              2750             17004               0.2
                             5              2758             17048               0.2
                             6              2766             16942               0.2
searchsorted(a,a)            2               506                55               9.2
                             3               708               490               1.4
                             4              3054              6507               0.5
                             5             32101             77762               0.4
                             6            378341            940912               0.4
sort(a)                      2               554                85               6.4
                             3               940               365               2.6
                             4              5394              3533               1.5
                             5             55238             36049               1.5
                             6            607845            420334               1.4
argsort(a)                   2              1541                90              16.9
                             3              2061               467               4.4
                             4              8113              5130               1.6
                             5             76924             49497               1.6
                             6            853114            588399               1.4
argmax(b)                    2           1112081              8692             127.9
                             3           1110896              9320             119.2
                             4           1108330              8706             127.3
                             5           1113449              8720             127.7
                             6           1112556              8743             127.3
array(s,typecode='i')        2               229               146               1.6
                             3               274              1103               0.2
                             4               764             10900               0.1
                             5              5450            105774               0.1
                             6             51500           1253819               0.0
dot(b,b)                     2           8825828          31402493               0.3
                             3           8804230          32136602               0.3
                             4           8748239          31123564               0.3
                             5           8811900          32910953               0.3
                             6           8827314          31784649               0.3
clip(a,50,100)               2              2920               177              16.4
                             3              3089               370               8.3
                             4              5138              2471               2.1
                             5             24340             24737               1.0
                             6            211277            245540               0.9
indices((100,100))           2              5253              2696               1.9
                             3              5230              2634               2.0
                             4              5290              2687               2.0
                             5              5383              2732               2.0
                             6              5558              2733               2.0
swapaxes(b,0,-1)             2                77                79               1.0
                             3                77                82               1.0
                             4                94               106               0.9
                             5               154               142               1.1
                             6               146               144               1.0
concatenate((a,a,a))         2              1021                35              29.1
                             3              1082                48              22.1
                             4              1868               431               4.3
                             5              8843              4945               1.8
                             6             75181             47495               1.6
innerproduct(b,b)            2           8719124           8989714               1.0
                             3           8662431           8840564               1.0
                             4           8734306           9033604               1.0
                             5           8694994           8935343               1.0
                             6           8680599           8869750               1.0
outerproduct(b[0],b[0])      2             42320             27598               1.5
                             3             42299             27802               1.5
                             4             42842             27935               1.5
                             5             42346             27822               1.5
                             6             42257             27709               1.5
resize(a,(2*10**i,))         2              1708               160              10.6
                             3              1769               171              10.3
                             4              2648               468               5.6
                             5              9778              3515               2.8
                             6             75513             31778               2.4
a[0]                         2                10                10               1.0
                             3                10                10               1.0
                             4                13                12               1.1
                             5                25                20               1.3
                             6                25                22               1.1
b[0]                         2                46                15               3.1
                             3                48                17               2.8
                             4                56                18               3.0
                             5               106                33               3.1
                             6               100                35               2.9
a[1000:2000]                 2               303                16              18.0
                             3               298                15              18.7
                             4               275                20              13.8
                             5               354                33              10.4
                             6               344                34               9.8
b[400:600,400:600]           2               320                25              12.8
                             3               317                25              12.2
                             4               348                30              11.3
                             5               432                46               9.2
                             6               418                47               8.7
arange(10.**i)               2               411                37              11.1
                             3               448                74               6.0
                             4               923               570               1.6
                             5              5630              5592               1.0
                             6             50735             54222               0.9
identity(i+2)                2              3147               226              13.9
                             3              3158               227              13.9
                             4              3214               258              12.4
                             5              3311               294              11.3
                             6              3316               291              11.4
add(a,c)                     2                76                27               2.7
                             3                90                40               2.2
                             4               313               225               1.4
                             5              2902              2128               1.4
                             6             28106             19972               1.4
add(a,c,d)                   2                40                22               1.9
                             3                46                33               1.4
                             4               144               132               1.1
                             5              1128              1165               1.0
                             6             10316             10827               1.0
a+c                          2                91                31               2.9
                             3               105                45               2.4
                             4               335               231               1.5
                             5              2945              2130               1.4
                             6             28102             19744               1.4
cos(a)                       2                84                53               1.6
                             3               212               235               0.9
                             4              1546              2351               0.7
                             5             15237             23074               0.7
                             6            151604            227898               0.7
add.reduce(a)                2                97                31               3.0
                             3               109                41               2.7
                             4               207               123               1.7
                             5              1062               949               1.1
                             6              8974              8844               1.0
add.reduce(b,0)              2             32527              9218               3.5
                             3             31652              9238               3.4
                             4             31376              9171               3.4
                             5             31412              9271               3.4
                             6             30443              9263               3.3
add.reduce(b,-1)             2              9065              9389               1.0
                             3              9075              9360               1.0
                             4              9102              9271               1.0
                             5              9168              9389               1.0
                             6              9148              9394               1.0
add.accumulate(a)            2               100                28               3.5
                             3               120                41               2.9
                             4               416               254               1.6
                             5              3584              2696               1.3
                             6             34837             25608               1.4
add.accumulate(b,0)          2             63947             25673               2.5
                             3             65875             25691               2.6
                             4             65540             25708               2.5
                             5             65161             25776               2.5
                             6             66539             25646               2.6
add.accumulate(b,1)          2             35048             25756               1.4
                             3             35022             25738               1.4
                             4             35048             25688               1.4
                             5             35092             25761               1.4
                             6             35110             25686               1.4






More information about the Numpy-discussion mailing list