[Numpy-discussion] .transpose() of memmap array fails to close()

Anne Archibald peridot.faceted@gmail....
Wed Aug 15 19:50:28 CDT 2007


On 15/08/07, Glen W. Mabey <Glen.Mabey@swri.org> wrote:
> On Tue, Aug 14, 2007 at 12:23:26AM -0400, Anne Archibald wrote:
> > On 13/08/07, Glen W. Mabey <Glen.Mabey@swri.org> wrote:
> >
> > > As I have tried to think through what should be the appropriate
> > > behavior for the returned value of __getitem__, I have not been able to
> > > see an appropriate solution (let alone know how to implement it) to this
> > > issue.
> >
> > Is the problem one of finalization? That is, making sure the memory
> > map gets (flushed and) closed exactly once? In this case the
> > numpythonic solution is to have only the original mmap object do any
> > finalization; any slices contain a reference to it anyway, so they
> > cannot be kept after it is collected. If the problem is that you want
> > to do an explicit close/flush on a slice object, you could just always
> > apply the close/flush to the base object of the slice if it has one or
> > the slice itself if it doesn't.
>
> The immediate problem is that when a numpy.memmap instance is created as
> another view of the original array, then __del__ on that new view fails.

Yes, this is definitely broken.

> flush()ing and closing aren't an issue for me, but they can't be
> performed at all on derived views right now.  It seems to me that any
> derived view ought to be able to flush(), and ideally in my mind,
> close() would be called [automatically] only just before the reference
> count gets decremented to zero.
>
> That doesn't seem to match the numpythonic philosophy you described,
> Anne, but seems logical to me, while still allowing for both manual
> flush() and close() operations.

You have to be a bit careful, because a view really is just a view
into the array - the original is still around. So you can't really
delete the array contents when the view is deleted. Really, if you do:
B = A[::2]
del B
nothing at all should happen to A.

But to be pythonic, or numpythonic, when the original A is
garbage-collected, the garbage collection should certainly close the
mmap.

Being able to apply flush() or whatever to slices is not necessarily
unpythonic, but it's probably a lot simpler to reliably implement
slices of mmap()s as simple slices of ordinary arrays. It means you
need to keep the original mmap object around (or traverse up the tree
of bases:
T = A
while T.base is not None: T = T.base
T.flush()
)

(Note that this would be simpler if when you did
A = arange(100)
B = A[::2]
C = B[::2]
you found that C.base were A rather than B.)

Anne


More information about the Numpy-discussion mailing list