[SciPy-user] Another memory quirk

Travis E. Oliphant oliphant@enthought....
Sat Nov 3 01:57:49 CDT 2007


Anand Patil wrote:
> Hi all,
>
> Not sure if this is related to my last post but the script below leaks 
> memory like mad... PLEASE DON'T RUN IT, I don't want to be responsible 
> for crashing your computer!
>
> Is there any safe way to store a ravelled view of an array in a 
> subclass like this? I'm using the ravelled view inside a very tight 
> loop & would rather not spend time calling ravel() if possible.
>
> Many thanks in advance,
> Anand

Here you have created another cyclic reference.

A.ravel() returns a new array object which references the old one (in 
the .base attribute).  In this case you are storing this new array 
object into the ._ravelled attribute of the old one.  Thus, the old 
MyArray object (the one returned from the view statement) is being 
referenced indirectly by-itself through its "_ravelled" attribute.   The 
reference count never goes to zero on the MyArray object and these 
cycles just blossom off each holding on to their original  ~8 MB array.

You've just shown two good examples where object cycles using arrays can 
hurt you.   It's an important point.   It may be possible to add cyclic 
garbage collection to NumPy objects (and it's sub-classes),  but it 
won't be trivial).

In this case, just keep the original array reference and set the 
_ravelled attribute to a raveled version of the original array:

B = zeros((1000,1000), float)
A = B.view(MyArray)
A._ravelled = B.ravel()

This way A is not referencing itself through one of its attributes and 
will be deleted once A gets re-assigned.

-Travis









>
>
> from numpy import *
> from PyMC import *
>    
> class MyArray(ndarray):
>     pass
>
> for i in xrange(1000):
>    
>     A=zeros((1000,1000),dtype=float)
>    
>     A = A.view(MyArray)
>
>     # If I uncomment this and run it, a huge memory leak ensues.
>     # A._ravelled = A.ravel()
>    
> ------------------------------------------------------------------------
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user@scipy.org
> http://projects.scipy.org/mailman/listinfo/scipy-user
>   



More information about the SciPy-user mailing list