[SciPy-dev] Linalg2 benchmarks

[Jochen Küpper]

Here are results from the same dual CPU PIII/800 machine.  ATLAS
compiled with gcc-2.7.2.3, everything else with gcc-3.

I just got the latest numpy and scipy cvs's and rebuild (--force) all.
See comments below.

,----
| Python 2.2.1c1 (#1, Mar 20 2002, 15:04:50) 
| [GCC 3.0.4] on linux2
| Type "help", "copyright", "credits" or "license" for more information.
| >>> import scipy
| >>> import scipy.linalg2
| >>> scipy.linalg2.basic.test()
| ................................
|            Finding matrix determinant
|       ==================================
|       |    contiguous     |   non-contiguous 
| ----------------------------------------------
|  size |  scipy  | Numeric |  scipy  | Numeric
|    20 |   0.36  |   0.74  |   0.36  |   0.86     (secs for 2000 calls)
|   100 |   0.83  |   1.50  |   0.83  |   2.01     (secs for 300 calls)
|   500 |   0.92  |   1.12  |   0.90  |   1.32     (secs for 4 calls)
| .
|       Solving system of linear equations
|       ==================================
|       |    contiguous     |   non-contiguous 
| ----------------------------------------------
|  size |  scipy  | Numeric |  scipy  | Numeric
|    20 |   0.53  |   0.64  |   0.53  |   0.75     (secs for 2000 calls)
|   100 |   0.89  |   1.22  |   0.89  |   1.77     (secs for 300 calls)
|   500 |   0.91  |   1.00  |   0.89  |   1.21     (secs for 4 calls)
| .
|            Finding matrix inverse
|       ==================================
|       |    contiguous     |   non-contiguous 
| ----------------------------------------------
|  size |  scipy  | Numeric |  scipy  | Numeric
|    20 |   0.76  |   1.17  |   0.76  |   1.28     (secs for 2000 calls)
|   100 |   2.22  |   3.63  |   2.22  |   4.03     (secs for 300 calls)
|   500 |   2.38  |   3.30  |   2.35  |   3.50     (secs for 4 calls)
| .
| ----------------------------------------------------------------------
| Ran 35 tests in 51.051s
| 
| OK
`----


On Fri, 5 Apr 2002 09:01:21 -0600 (CST) pearu  wrote:

pearu> On 4 Apr 2002, Jochen Küpper wrote:

>> I don't know what the problem is, but it looks as scipy scales worse
>> than Numeric?

pearu> I don't understand how can you conclude that 

Well, se above.  Going from 300 x 100x100 to 4 x 500x500 scipy takes
more or the same time, whereas numpy takes less.

pearu> but if scipy and Numeric use the same ATLAS (the Jochen case)
pearu> then:

pearu> 1) for n->oo, where n is the size of the problem, there would be no
pearu> difference in speeds as the hard computation is done by the same ATLAS
pearu> routine.

The data does not oppose that.

pearu> 2) for n fixed but repeating the computation for c times, then for c->oo
pearu> you would find that scipy is 2-3 times faster that Numeric. This speed up
pearu> is gained only because of the f2py generated interface between Python and
pearu> the ATLAS routines that scipy uses.

That is kind of what the data shows, but is only valid for small n
though, as you said yourself in 1).

pearu> BUT, if Numeric is linked with its lapack_lite (the Travis
pearu> case), then you will have huge speedups (approx. 10 times)
pearu> mainly because scipy uses highly optimized ATLAS routines.

Ok.

pearu> So, I don't find these testing results strange as you commented.

I think it is not a valid comparison to put scipy against lapack_lite,
considering how easy it is to get numpy use any LAPACK/BLAS.
Esp. considering the people on this list.

pearu> More surprising is that sometimes with non-contiguous input
pearu> data the calculation is actually faster(!) and not slower as I
pearu> would expect. I have no explanation for this one.

I would assume that gives you a lower bound for the accuracy of these
benchmarks... 

Greetings,
Jochen
-- 
University of North Carolina                       phone: +1-919-962-4403
Department of Chemistry                            phone: +1-919-962-1579
Venable Hall CB#3290 (Kenan C148)                    fax: +1-919-843-6041
Chapel Hill, NC 27599, USA                            GnuPG key: 44BCCD8E