[SciPy-user] shared memory machines

Sturla Molden sturla@molden...
Mon Feb 2 11:41:32 CST 2009


> On Mon, Feb 2, 2009 at 04:53, Gael Varoquaux

> Almost. On Windows, the subprocesses inherit nothing. All objects must
> be passed through pickles. Passing the Array works, but passing the
> ndarray won't because the ndarray pickler will pass-by-value.

Almost. A subprocess can be specified to inherit its parent's handles. The
parent must then pass the value of the handle to the subprocess, e.g. via
the  stdin pipe. This is how mp.Array works on Windows.

> On UNIX, Arrays and the stuff underneath it don't pickle because the
> underlying mmap is not named.

It is the same on Windows. Named shared memory is the cure in both cases.

The advantage of named shared memory is that it can be created after the
subprocesses are spawned/forked.

Sturla Molden



More information about the SciPy-user mailing list