[SciPy-user] lingalg.svd performance?

Pearu Peterson pearu at scipy.org
Sat Oct 16 01:02:37 CDT 2004


On Fri, 15 Oct 2004, Nick Arnett wrote:

> What sort of performance should I expect using lingalg.svd on matrixes
> on the order of 500 x 15000?  I'm using a 2 Ghz P4 system with 2 GB of
> memory and these are taking hours to complete.  I previously was using
> an external SVD utility that was taking a couple of minutes, and I'm
> going to have to switch back to it unless there's some way to get Scipy
> to do this a lot faster.  I'm using the pre-built binaries, with ATLAS
> and F2PY installed.  I'm not eager to build my own, but if that is
> likely to speed things up a lot, I'll give it a shot.

On a system with AMD Athlon(tm) 64 Processor 3000+ and 2GB memory:

- calculating full svd of a 500x15000 matrix takes 5m55s consuming about
   1.8g memory
- calculating only singular values takes 22s consuming about 0.2g memory

Note that the above results do not contain time spent for swaping when 
memory is low. E.g. on a 1GB system calculating full svd would take 
forever..

HTH,
Pearu

Python 2.3.3 (#1, Aug 28 2004, 06:56:51)
[GCC 3.3.3 20040412 (Gentoo Linux 3.3.3-r6, ssp-3.3.2-2, pie-8.7.6)] on 
linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from scipy import *
>>> from scipy_test.testing import jiffies
>>> t=jiffies();s=linalg.svd(rand(500,15000),compute_uv=0); print 
(jiffies()-t)/100,'sec'
22 sec
>>> t=jiffies();u,s,vh=linalg.svd(rand(500,15000),compute_uv=1); print 
(jiffies()-t)/100,'sec'
345 sec
>>> import scipy.linalg.atlas_version
ATLAS version 3.7.8 built by pearu on Thu Sep 30 23:29:06 EEST 2004:
    UNAME    : Linux opt 2.6.3-gentoo-r2 #1 SMP Sat Aug 28 12:40:32 EEST 
2004 x86_64 AMD Opteron(tm) Processor 244 AuthenticAMD GNU/Linux
    INSTFLG  :
    MMDEF    : 
/usr/local/src/ATLAS-3.7.8/CONFIG/ARCHS/HAMMER64SSE2/gcc/gemm
    ARCHDEF  : 
/usr/local/src/ATLAS-3.7.8/CONFIG/ARCHS/HAMMER64SSE2/gcc/misc
    F2CDEFS  : -DAdd_ -DStringSunStyle
    CACHEEDGE: 917504
    F77      : /usr/x86_64-pc-linux-gnu/gcc-bin/3.3/g77, version GNU 
Fortran (GCC) 3.3.4 20040623 (Gentoo Linux 3.3.4-r1, ssp-3.3.2-2, 
pie-8.7.6)
    F77FLAGS : -fomit-frame-pointer -O -m64 -fno-second-underscore -fPIC
    CC       : /usr/bin/gcc, version gcc (GCC) 3.3.4 20040623 (Gentoo Linux 
3.3.4-r1, ssp-3.3.2-2, pie-8.7.6)
    CC FLAGS : -fomit-frame-pointer -O -mfpmath=387 -m64 -fPIC
    MCC      : /usr/bin/gcc, version gcc (GCC) 3.3.4 20040623 (Gentoo Linux 
3.3.4-r1, ssp-3.3.2-2, pie-8.7.6)
    MCCFLAGS : -fomit-frame-pointer -O -mfpmath=387 -m64 -fPIC



More information about the SciPy-user mailing list