[Numpy-discussion] why std() eats much memory in multidimensional case?

Emanuele Olivetti emanuele@relativita....
Fri Apr 20 09:57:13 CDT 2007


Hi,
I'm working with 4D integer matrices and need to compute std() on a
given axis but I experience problems with excessive memory consumption.
Example:
---
import numpy
a = numpy.random.randint(100,size=(50,50,50,200)) # 4D randint matrix
b = a.std(3)
---
It seems that this code requires 100-200 Mb to allocate 'a'
as a matrix of integers, but requires >500Mb more just to
compute std(3). Is it possible to compute std(3) on integer
matrices without spending so much memory?

I manage 4D matrices that are not much bigger than the one in the example
and they require >1.2Gb of ram to compute std(3) only.
Note that quite all this memory is immediately released after
computing std() so it seems it's used just internally and not to
represent/store the result. Unfortunately I haven't all that RAM...

Could someone explain/correct this problem?

Thanks in advance,

Emanuele



More information about the Numpy-discussion mailing list