[SciPy-User] Help: Multiplying two sparse matrices

Sebastian Berg sebastian@sipsolutions....
Tue Dec 25 12:47:06 CST 2012


On Tue, 2012-12-25 at 19:34 +0100, Pranava Swaroop Madhyastha wrote:
> Hi,
> 
> I have a problem multiplying two sparse matrices. Here is a sample:
> 
> import scipy.sparse as ss
> 
> D = ss.rand(49837, 1000, format='csr', density=0.1)
> 
> E = ss.rand(1000, 49837, format='csr', density=0.1)
> 
> D * E
> 
> This is the error that I get:
> 
> ---------------------------------------------------------------------------
> ValueError                                Traceback (most recent call last)
> <ipython-input-4-c94d439b5bf5> in <module>()
> ----> 1 D * E
> 
> /usr/local/lib/python2.7/dist-packages/scipy/sparse/base.pyc in
> __mul__(self, other)
>     254             if self.shape[1] != other.shape[0]:
>     255                 raise ValueError('dimension mismatch')
> --> 256             return self._mul_sparse_matrix(other)
>     257
>     258         try:
> 
> /usr/local/lib/python2.7/dist-packages/scipy/sparse/compressed.py in
> _mul_sparse_matrix(self, other)
>     296
>     297         nnz = indptr[-1]
> --> 298         indices = np.empty(nnz, dtype=np.intc)
>     299         data    = np.empty(nnz, dtype=upcast(self.dtype,other.dtype))
>     300
> 
> ValueError: negative dimensions are not allowed
> 

This is because the result has too many nonzero elements. Scipy seems to
use 32bit integers to save positions, and if you have too many non-zero
elements, 32bit integers can overflow easily... Maybe if you feel like
it, you could do a PR against scipy to change the error message with
something understandable. I don't know if there is a great reason to use
32bit integers, though I guess it saves quite a bit of RAM if it is
large enough.

However, if your result has so many nonzero elements, it seems sounds
like the sparsity is probably so low, you probably could just as well
use dense matrix (for the result, if possible), for this example.

Regards,

Sebastian

> 
> Apparently, the error seems to be because of dtype=np.intc. Is there a
> work around for this?
> 
> Thanks and Regards,
> Pranava
> _______________________________________________
> SciPy-User mailing list
> SciPy-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
> 




More information about the SciPy-User mailing list