[SciPy-User] scipy.sparse.csr_matrix: Refills with same graph, different values

Dag Sverre Seljebotn dagss@student.matnat.uio...
Wed Oct 13 01:01:07 CDT 2010


On 10/12/2010 12:36 PM, Robert Cimrman wrote:
> On Tue, 12 Oct 2010, Nico Schlömer wrote:
>
>>> Finite element assembling, right?
>>
>> Finite volumes, but yeah, the structure is really similar.
>
> Oh yes.
>
>>> It seems that some of the entries you expect to exist do not exist. The
>>> following works for me (scipy 0.9.0.dev6812):
>>
>> Correct. At the creating of the lil_matrix, some of the values were in
>> fact 0 and got stripped upon .tocsr(). I can exclude this case now and
>> run the code without the performance warning.
>> Still, it runs a lot slower when the csr_matrix is filled directly.
>> The following snippet may help making clear what I'm talking about:
>> The second loop takes about twenty (!) times as long when running this
>> on my computer.
>
> Same on mine. IMHO it's because element access in CS* matrices is 
> rather complex to allow some fancy indexing and slices, so accessing a 
> single element has a significant overhead. LIL, on the other hand is 
> designed to behave well with single element access.

This may be because of Python overhead in this case. For instance, 
there's this line leading to allocating a new temporary boolean array 
(__setitem__ in compressed.py):

             indxs = np.where(minor_index == self.indices[start:end])[0]

Cythonizing the item assignment code (and throwing in the special cases 
for CSC/CSR in another way) may give back that factor of twenty...this 
could be done in an external routine as well, no need to modify SciPy, 
just operate on the data, indices, indptr arrays.

Dag Sverre


>
> r.
>
>> =================================== *snip* 
>> ===================================
>> # -*- coding: utf-8 -*-
>>
>> from scipy import sparse
>> import time
>>
>> n = 10000
>>
>> # 
>> ------------------------------------------------------------------------------ 
>>
>> starttime = time.clock()
>> A = sparse.lil_matrix( ( n, n ) )
>> for k in range( 1, n-1 ):
>>    A[k,k-1] -= 0.5
>>    A[k,k]   += 1.0
>>    A[k,k+1] -= 0.5
>> A = A.tocsr()
>> endtime = time.clock()
>> print endtime - starttime
>> # 
>> ------------------------------------------------------------------------------ 
>>
>> A.data[:] = 0.0
>> starttime = time.clock()
>> for k in range( 1, n-1 ):
>>    A[k,k-1] -= 0.5
>>    A[k,k]   += 1.0
>>    A[k,k+1] -= 0.5
>> endtime = time.clock()
>> print endtime - starttime
>> # 
>> ------------------------------------------------------------------------------ 
>>
>> =================================== *snap* 
>> ===================================
>
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>    

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/scipy-user/attachments/20101013/0823096e/attachment.html 


More information about the SciPy-User mailing list