[SciPy-user] Sparse matrices and weave.inline, f2py or Pyrex
schofield at ftw.at
Sun Aug 13 08:37:47 CDT 2006
> On 11/08/06, David Grant <davidgrant at gmail.com> wrote:
>> On 8/11/06, William Hunter <willemjagter at gmail.com> wrote:
>> > List;
>> > Has anyone ever attempted using sparse matrices in combination with
>> > either one of the following; weave.inline, f2py or Pyrex?
>> > What is the best option? Perhaps a case of personal preference?
>> > Reason I'm asking: I've gotten the solving part of my code to run
>> > satisfactorily, but building them with Python's for loops takes
>> a heck
>> > of a long time, too long.
>> Have you tried using vector indexing with numpy first? See if you
>> can remove
>> some of those python for loops.
On 12/08/2006, at 1:50 PM, William Hunter wrote:
> Yes, well, my best attempt at it. You'll notice that the sparse
> matrices doesn't support fancy indexing, unfortunately. (I'm hoping
> someone can prove me wrong :-)
> Here's some code you can fiddle with (also attached -
> sparsefiddle.py), if you want.
> [code snipped]
Yeah, sparse fancy indexing is still only partially supported. I
don't think it'll ever be possible to make sparse support all of
NumPy's indexing tricks. But I'd like to fill out the sparse indexing
code so that more NumPy-like indexing operations are supported; this
would at least improve usability. We could also definitely improve
performance by a constant factor by taking the loops inside the
methods such as __setitem__. You could try hacking the
lil_matrix.__setitem__ method to accept index sequences of the form
returned by ix_ to avoid all the extra method calls. (And I'd be
happy to accept a patch!)
It might be possible to speed up your code somewhat by doing the
loops in a compiled language, but I expect this would be a complex
endeavour and the gains would be small. To get speed gains of orders
of magnitude you'd need to think about the optimal sparse data type
and find an algorithm that's suitable for your matrix structure. I
think constructing sparse matrices efficiently is inherently a tricky
More information about the SciPy-user