[Numpy-discussion] Error when multiplying large sparse matrices

[Jeremy Lecoeur]

Hi,

I have been using the sparse matrix tools for a while to do all sort of 
things and, using the same code that was working just fine, I now 
encounter a problem when trying . I do have very large sparse matrices 
and when i multiplying them the number of non zeros exceed the max value 
of an intc, which cause indptr to hold negative values. Hence in the 
multiplication function of csr, whenc reating the resulting matrix, i 
get an error as it is not possible to have a negative value for a matrix 
size.
Am I missing something that would allow me to do that computation ?

Here is the code I am using:

def main():

   inCondMatFile      = sys.argv[1]

   inNodeSize         = sys.argv[2]

   outProfileFile     = sys.argv[3]

   outNodeDistFile    = sys.argv[4]

   outNodeDensityFile = sys.argv[5]

   Acsr = scipy.io.mmread(inCondMatFile).tocsr().sorted_indices()

   A = Acsr.tocoo()

   n = A.shape[0]

   nnz = A.nnz

   rows=numpy.zeros(3*nnz+n, dtype=numpy.int32)

   cols=numpy.zeros(3*nnz+n, dtype=numpy.int32)

   data=numpy.zeros(3*nnz+n, dtype=numpy.float64)

   #first n rows of constraint mat is A - I

   rows[0:nnz] = A.row

   cols[0:nnz] = A.col

   data[0:nnz] = A.data

   rows[nnz:nnz+n] =  numpy.arange(n)

   cols[nnz:nnz+n] =  numpy.arange(n)

   data[nnz:nnz+n] = -numpy.ones(n)

   #rows n to n+nnz are

   #A_{i,j} d_{j} - A_{j,i} d_{i} == 0

   rows[nnz+n:] = numpy.append(numpy.arange(n,n+nnz),numpy.arange(n,n+nnz))

   cols[nnz+n:] = numpy.append(A.col,A.row)

   data[nnz+n:] = numpy.append(A.data,-Acsr[A.col,A.row])

   tmpC = scipy.sparse.coo_matrix( (data, (rows,cols) ) )

   Ptmp = (tmpC.transpose().tocsr() * tmpC.tocsr()).tocoo()


And it fails for that last multiplication (i did not include the rest of 
the code) because of an nnz way too big for an intc.

Jeremy



The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.