[Numpy-discussion] dimensions too large error

Stéfan van der Walt stefan@sun.ac...
Fri Mar 14 21:52:56 CDT 2008


Hi Dinesh

On Fri, Mar 14, 2008 at 6:00 PM, Dinesh B Vadhia
<dineshbvadhia@hotmail.com> wrote:
> For the following code:
>
> I = 18000
> J = 33000
> filename = 'ij.txt'
> A = scipy.asmatrix(numpy.empty((I,J), dtype=numpy.int))
>     for line in open(filename, 'r'):
>         etc.
>
> The following message appears:
>
> Traceback (most recent call last):
>     File "C:\...\....py", line 362, in <module>
>     A= scipy.asmatrix(numpy.empty((I,J), dtype=numpy.int))
>     ValueError: dimensions too large.
>
> Is there a limit to array/matrix dimension sizes?

You are trying to allocate a contiguous block of memory of roughly
2.2Gb.  I'm wondering whether you have enough memory available, and
whether that memory is not already fragmented?

If your matrix is not dense, you can use the sparse matrix structures
from scipy.sparse to represent all the non-zeros.

Regards
Stéfan


More information about the Numpy-discussion mailing list