[SciPy-user] trouble saving sparse matrix

Robin robince@gmail....
Wed Nov 26 10:51:59 CST 2008


Hi,

I have a large sparse matrix (about 9GB):

In [18]: a.A
Out[18]:
<21699x1048575 sparse matrix of type '<type 'numpy.int8'>'
        with 1035272192 stored elements in Compressed Sparse Column format>

but I am having trouble saving it.

I am on 64 bit linux.

The problem is whatever I try I get :
SystemError: Negative size passed to PyString_FromStringAndSize

This happens with cPickle.dump, np.save, sp.io.savemat etc.

I am guessing something is overflowing a 32 bit integer. The matrix
itself seems ok... I was wondering if anyone had any ideas for another
way to save it - or perhaps if I have made a mistake and it is really
too big?

Cheers

Robin


More information about the SciPy-user mailing list