[SciPy-user] sparse csr_matrix memeory error

Nathan Bell wnbell@gmail....
Sun Aug 10 17:39:13 CDT 2008


On Sun, Aug 10, 2008 at 5:29 PM, Dinesh B Vadhia
<dineshbvadhia@hotmail.com> wrote:
> I'm obtaining a memory error when creating a large sparse csr matrix as
> follows:
>
> I = 680000
> J = 900000
> nnz = 72000000
> row = numpy.empty(nnz, dtype='intc')
> column = numpy.empty(nnz, dtype='intc')
> # read (i,j) data into row and column
> data = scipy.ones(nnz, dtype='intc')

Together, the arrays for row,column, and data take 864MB of memory.
You need approximately 2x that to do the conversion to CSR.

> I'm running the program under Windows XP with over 2gb memory.  Any thoughts
> on what the problem is?

Yes, your matrix is simply too large.

-- 
Nathan Bell wnbell@gmail.com
http://graphics.cs.uiuc.edu/~wnbell/


More information about the SciPy-user mailing list