[SciPy-user] shared memory machines

Gael Varoquaux gael.varoquaux@normalesup....
Mon Feb 9 05:38:35 CST 2009


On Mon, Feb 09, 2009 at 12:28:59PM +0100, Sturla Molden wrote:
> On System V IPC a shared segment it has to be marked for removal, i.e. 
> there are no reference counting in the kernel as in Windows. So I was 
> thinking out marking it for removal when the attachment count is zero. 
> But as you have used Posix V IPC I have no idea. Just make sure it does 
> not produce a global memory leak.

Hum, I believe you are right, and I have produced just that.

This means that we probably need a shared reference counter :(. Sounds
tedious to implement. Do people have any suggestions on how to implement
this? I can see several possibilities: 

    * Using multiprocessing to share the dictionnary of shared map
      addresses, but this induces a tight coupling with multiprocessing, and 
      I am not sure we want this.

    * Sharing this dictionnary via a C structure, ie to do our own
      implementation of a shared state.

    * Add the ref count information in the shared array. For instance the
      first byte could be the ref count. This sounds the easiest option,
      but I am probably not seeing some of the problems that will arize
      from this approach.

I think I am going to take a stab at option three, tonight or later in
the week, but please, wise people of the list, give me feedback on what
you think might work.

Gaël


More information about the SciPy-user mailing list