[SciPy-user] combine two sparse matrices
Robin
robince@gmail....
Sat May 3 10:47:37 CDT 2008
On Sat, May 3, 2008 at 4:24 PM, Nathan Bell <wnbell@gmail.com> wrote:
>
> If you're worried about speed or memory consumption, then you should
> avoid lil_matrix and dok_matrix.
>
> The fastest and most memory efficient approach uses coo_matrix:
>
> row = array([2,3,1,4])
> col = array([4,2,3,1])
> data = array([5,5,5,5])
> A = coo_matrix( (data,(row,col)), shape=(5,5)).tocsr()
>
> The conversion to CSR forces a copy, however CSR and COO are so much
> more memory efficient than LIL/DOK that it shouldn't matter.
Thanks - I was just thinking about COO matrices. The problem is I have
to build it up in parts - I know lists are more efficient to extend
than arrays so I could store the data, row, col as lists and extend
with each chunk - then I think I can construct the COO matrix from the
list. I was worried that the lists would stay around though - even if
deleted so then I wouldn't have enough memory to do the COO-CSC
conversion.
Perhaps I could add another layer of indirection with another fork()
though to make sure the lists are cleaned up and just pass back the
COO matrix - hopefully the COO matrix will pickle OK.
Thanks,
Robin
More information about the SciPy-user
mailing list