[SciPy-user] mid-size outerproduct too big for memory

Robert Kern rkern at ucsd.edu
Sun Apr 24 22:38:56 CDT 2005

Thomas Davidoff wrote:
> I tried to get the outerproduct(a,b) where a is a 23,297 by 15 matrix 
> and b is a 15 element row vector.  This led to a memory error.  This is 
> a pretty small dataset for empirical economics - is there a way around 
> this? 

23297*15*15*8./1024/1024 = 40

40 MB. My G4-1GB memory handles this fine. I'm not sure why you are 
seeing a memory error. Could you post your code?

> Will I ever be able to do something like 1 million by 100 or is 
> it back to Stata for me?

1000000*100*100*8/1024/1024/1024 == 74

74 GB. No, I don't think your machine will handle that outerproduct well.

> I don't think the issue is other stuff in the program because nothing 
> should be eating a lot of memory. 

How about other stuff running on the system?

> I noticed I get the same problem in 
> command line python just trying a 100,000 by 100 matrix and a 1 by 100 
> row vector.
> I am on a new G5 PowerMac with lots of new memory installed.

Robert Kern
rkern at ucsd.edu

"In the fields of hell where the grass grows high
  Are the graves of dreams allowed to die."
   -- Richard Harter

More information about the SciPy-user mailing list