[Numpy-discussion] numpy/Windows shared arrays between processes?
Tue Oct 9 02:19:29 CDT 2007
I was in fact experimenting with this. The solution seemed to lie in
"simple" memmap as it is implemented in Windows:
import numpy as N
def arrSharedMemory(shape, dtype, tag="PriithonSharedMemory"):
Windows only !
share memory between different processes if same `tag` is used.
itemsize = N.dtype(dtype).itemsize
count = N.product(shape)
size = count * itemsize
sharedmem = mmap.mmap(0, size, tag)
a=N.frombuffer(sharedmem, dtype, count)
a.shape = shape
For explaintion look up the microsoft site for the mmap documentation.
And/or the Python-doc for mmap.
(( I have to mention, that I could crash a process while testing this ... ))
If anyone here would know an equivalent way of doing this on
Linux/OS-X we were back to a cross-platfrom function.
Hope this helps,
On 10/9/07, David Cournapeau <email@example.com> wrote:
> Ray S wrote:
> > Is anyone sharing arrays between processes on Windows?
> > I tried compiling the posh sources (once, so far) with the new MS
> > toolkit and failed...
> > What other solutions are in use?
> > Have a second process create an array view from an address would
> > suffice for this particular purpose. I could pass the address as a
> > parameter of the second process's argv.
> > I've also tried things like
> > pb=pythonapi.PyBuffer_FromReadWriteMemory(9508824, 9*sizeof(c_int))
> > N.frombuffer(pb, N.int32)
> > which fails since pb is and int. What are my options?
> (disclaimer: I know nothing about windows idiosyncraties)
> Could not this be because you compiled the posh sources with a
> compiler/runtime which is different than the other extensions and python
> interpreter ? I don't know the details, but since most of the posix
> functions related to files and processes are broken beyond despair in
> windows, and in particular, many posix handles cannot cross dll
> boundaries compiled by different compilers, I would not be surprised if
> this cause some trouble.
> The fact that POSH is said to be posix-only on python.org
> (http://wiki.python.org/moin/ParallelProcessing) would imply that people
> do not care much about windows, too (but again, this is just from
> reading what posh is about; I have never used it personnally).
> Numpy-discussion mailing list
More information about the Numpy-discussion