[SciPy-User] SciPy-User Digest, Vol 111, Issue 20

Gabriel Gaster gabe@gabegaster....
Fri Nov 16 09:59:11 CST 2012


Sorry -- never mind -- this was figured out on StackOverflow.  It was actually an overflow error.

http://stackoverflow.com/questions/13402393/scipy-sparse-matrices-inconsistent-sums

--

Gabriel Gaster
Data Scientist
gabe@gabegaster.com, twitter, blog
773 816 3349, skype: gabrielgaster

On Nov 16, 2012, at 8:04 AM, scipy-user-request@scipy.org wrote:

> Send SciPy-User mailing list submissions to
> 	scipy-user@scipy.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> 	http://mail.scipy.org/mailman/listinfo/scipy-user
> or, via email, send a message with subject or body 'help' to
> 	scipy-user-request@scipy.org
> 
> You can reach the person managing the list at
> 	scipy-user-owner@scipy.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of SciPy-User digest..."
> 
> 
> Today's Topics:
> 
>   1. Sparse Matrices, summing columns !=sum (gabe.g@me.com)
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Thu, 15 Nov 2012 11:19:40 -0600
> From: gabe.g@me.com
> Subject: [SciPy-User] Sparse Matrices, summing columns !=sum
> To: scipy-user@scipy.org
> Message-ID: <49FBA830-EC6C-4C8A-8523-F1499BFC6E65@me.com>
> Content-Type: text/plain; charset="us-ascii"
> 
> Hi all, I'm new to this list. Have a specific question about Sparse matrices.
> 
> I posted this to StackOverflow and Anony-Mousse suggested it might be a bug and that I should post to this list:
> 
> So-- is this a bug or am I doing something wrong?
> Thanks much.
> Gabe
> 
> 
> I have a sparse matrix that I arrived at through a complicated bunch of calculations which I cannot reproduce here. I will try to find a simpler example of this.
> 
> For now, does anyone know how it might be (even remotely) possible that I could have a sparse matrix X with the property that:
> 
> In [143]: X.sum(0).sum()
> Out[143]: 131138
> 
> In [144]: X.sum()
> Out[144]: 327746
> 
> In [145]: X.sum(1).sum()
> Out[145]: 327746
> 
> In [146]: type(X)
> Out[146]: scipy.sparse.csr.csr_matrix
> My only guess is that if I want to sum columns correctly, I need to first cast the matrix as csc -- which makes sense. Although one would think that the sparse package would handle column sums gracefully (or throw an error) instead of just giving a WRONG answer.
> 
> After more thought, I tried the following:
> 
> In [164]: X.tocsr().sum(0).sum()
> Out[164]: 131138
> 
> In [165]: X.tocsc().sum(0).sum()
> Out[165]: 131138
> 
> In [166]: X.tocoo().sum(0).sum()
> Out[166]: 131138
> 
> In [167]: X.tolil().sum(0).sum()
> Out[167]: 131138
> 
> In [168]: X.todok().sum(0).sum()
> Out[168]: 131138
> 
> In [169]: X.shape
> Out[169]: (196980, 43)
> 
> In [170]: X
> Out[170]: 
> <196980x43 sparse matrix of type '<type 'numpy.uint16'>'
>        with 70875 stored elements in Compressed Sparse Row format>
> 
> In [172]: X.todense().sum(0)
> Out[172]: 
> matrix([[170726,   1041, 117398,   3526,  13202,   3585,   2355,   1895,   1392,   2189,   2070,   2603,   1676,    496,   1194,    933,    129,
>            529,    544,    256,      7,      0,      0,      0,      0,      0,      0,      0,      0,      0,      0,      0,      0,      0,
>              0,      0,      0,      0,      0,      0,      0,      0,      0]], dtype=uint64)
> 
> In [173]: X.sum(0)
> Out[173]: 
> matrix([[39654,  1041, 51862,  3526, 13202,  3585,  2355,  1895,  1392,  2189,  2070,  2603,  1676,   496,  1194,   933,   129,   529,   544,   256,
>             7,     0,     0,     0,     0,     0,     0,     0,     0,     0,     0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
>             0,     0,     0]], dtype=uint16)
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mail.scipy.org/pipermail/scipy-user/attachments/20121115/2a8bbe49/attachment.html 
> 
> ------------------------------
> 
> _______________________________________________
> SciPy-User mailing list
> SciPy-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
> 
> 
> End of SciPy-User Digest, Vol 111, Issue 20
> *******************************************

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/scipy-user/attachments/20121116/c8211a12/attachment.html 


More information about the SciPy-User mailing list