[SciPy-user] Sparse matrices and weave.inline, f2py or Pyrex

Ed Schofield schofield at ftw.at
Sun Aug 13 08:37:47 CDT 2006

> On 11/08/06, David Grant <davidgrant at gmail.com> wrote:
>> On 8/11/06, William Hunter <willemjagter at gmail.com> wrote:
>> > List;
>> >
>> > Has anyone ever attempted using sparse matrices in combination with
>> > either one of the following; weave.inline, f2py or Pyrex?
>> >
>> > What is the best option? Perhaps a case of personal preference?
>> >
>> > Reason I'm asking: I've gotten the solving part of my code to run
>> > satisfactorily, but building them with Python's for loops takes  
>> a heck
>> > of a long time, too long.
>> Have you tried using vector indexing with numpy first? See if you  
>> can remove
>> some of those python for loops.

On 12/08/2006, at 1:50 PM, William Hunter wrote:
> Yes, well, my best attempt at it. You'll notice that the sparse
> matrices doesn't support fancy indexing, unfortunately. (I'm hoping
> someone can prove me wrong :-)
> Here's some code you can fiddle with (also attached -
> sparsefiddle.py), if you want.
> [code snipped]

Yeah, sparse fancy indexing is still only partially supported. I  
don't think it'll ever be possible to make sparse support all of  
NumPy's indexing tricks. But I'd like to fill out the sparse indexing  
code so that more NumPy-like indexing operations are supported; this  
would at least improve usability. We could also definitely improve  
performance by a constant factor by taking the loops inside the  
methods such as __setitem__. You could try hacking the  
lil_matrix.__setitem__ method to accept index sequences of the form  
returned by ix_ to avoid all the extra method calls. (And I'd be  
happy to accept a patch!)

It might be possible to speed up your code somewhat by doing the  
loops in a compiled language, but I expect this would be a complex  
endeavour and the gains would be small. To get speed gains of orders  
of magnitude you'd need to think about the optimal sparse data type  
and find an algorithm that's suitable for your matrix structure. I  
think constructing sparse matrices efficiently is inherently a tricky  
business ...

-- Ed

More information about the SciPy-user mailing list