[SciPy-user] Arrays and strange memory usage ...

christophe grimault christophe.grimault@novagrid....
Wed Sep 3 04:49:50 CDT 2008


OK, I understand now ! 
Thanks very for the explanation.

Chris

On Wed, 2008-09-03 at 10:49 +0200, Francesc Alted wrote:
> A Tuesday 02 September 2008, christophe grimault escrigué:
> > Hi,
> >
> > I have a application that is very demanding in memory ressources. So
> > I started to to look closer at python + numpy/scipy as far as memory
> > is concerned.
> >
> > I can't explain the following :
> >
> > I start my python, + import scipy. A 'top' in the console shows that
> > :
> >
> >   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME COMMAND
> > 14791 grimault  20   0 21624 8044 3200 S    0  0.4   0:00.43 python
> >
> > Now after typing :
> >
> > z = scipy.arange(1000000)
> >
> > I get :
> > 14791 grimault  20   0 25532  11m 3204 S    0  0.6   0:00.44 python
> >
> > So the memory increased by ~ 7 Mb. I was expecting 4 Mb since the
> > data type is int32, giving 4*1000000 = 4 Mb of memory chunk (in C/C++
> > at least).
> 
> You should look at the "RES" column instead of "VIRT" one.  "RES" column 
> shows the *real* memory that you are consuming.  So, in this case, you 
> have consumed 11MB - 8044KB ~ 3 MB.  However, you are undergoing here 
> the effects of number representation truncation.  Your consumed memory 
> should be rather: 8044KB + 3906KB = 11950KB, but as it is converted to 
> MB (scale changes happens automatically in 'top' when the figures need 
> more than 4 digits to be represented), 11959KB is truncated and 950KB 
> are gone, so this is why the final figure you are seeing is 11MB.  This 
> can be a bit misleading at first sight, but be sure that your machine 
> (and NumPy) is doing fine and works as expected.
> 
> Cheers,
> 




More information about the SciPy-user mailing list