[SciPy-dev] advanced memory allocator?

[Robert Cimrman]

Hi all,

Following recent discussions here about malloc vs. python allocator and 
about hunting memory related problems like buffer overruns and memory 
leaks, I think a better memory allocator could help. Years ago I have 
looked at the allocator of PETSc and, inspired by it, wrote a much 
simpler but usable version, which can be used to replace malloc/free.

All it does is:
- for each data allocation it records the line, function, file and dir 
where it occurred, together with allocated size, in a header prepended 
to the data buffer
- record current/max. memory usage
- all currently dynamically allocated memory can be printed or freed 
(brutal version of garbage collection :)
- memory can be checked for integrity, thanks to a 'magic cookie' value 
in the header - many buffer overruns can be detected this way
- all you pay for this is a little extra space for each allocation call

Then any python extension module using the above can be at any moment 
examined and its allocated memory checked, printed, freed etc.

Of course, valgrind gives you much more, but this can be used all the 
time since it does not cause any performance problems...

There is one problem though: it is tied to 32-bit systems, so a 64-bit 
expert would have to take a look...

I will upload it into the sandbox, if some interest arises.

cheers,
r.