[SciPy-user] shared memory machines

Sturla Molden sturla@molden...
Fri Feb 6 09:15:05 CST 2009


On 2/6/2009 1:34 AM, Brian Granger wrote:

> Can these mechanisms be used to create shared memory amongst processes
> that are started in a completely independent manner.  That is, they
> are not fork()'d.

Yes it can. If we know the name of the segment (an integer on Unix, a 
string on Windows), it can be mapped into any process. Similarly for 
named semaphores.

This is different from the locks ans shared memory in multiprocessing, 
which must be shared through forking (Unix) or handle inheritance 
(Windows), and therefore created prior to instantiation of 
multiprocessing.Process. Otherwise, there is no valid handle to inherit.

The question remains: should we base this on Cython or C (there is very 
little coding to do), or some third party extension, e.g. Philip 
Semanchuk's POSIX IPC and Mark Hammond's pywin32? I am thinking that at 
least for POSIX IPC, GPL is a severe limitation. Also we need some 
automatic clean up, which can only be accomplished with an extension 
object (that is, __dealloc__ in Cython will always be called, as opposed 
to __del__ in Python). In Pywin32 there is a PyHANDLE object that 
automatically calls CloseHandle when it is collected. But I don't think 
Semanchuk's POSIX IPC module will do the same. And avoiding dependencies 
on huge projects like pywin32 is also good.

Sturla Molden


More information about the SciPy-user mailing list