[SciPy-user] shared memory machines

Andrew Straw strawman@astraw....
Thu Feb 5 22:13:34 CST 2009


FWIW, I wrote some BSD licensed Pyrex code that does some shared memory
stuff. I wouldn't attempt to resurrect the complete working module, but
cut and paste at will:

http://code.astraw.com/projects/motmot/browser/trunk/pycamiface/src/_cam_iface_shm.pyx?rev=328

(This was from a wrapper of a camera driver that used shared memory
since the camera driver was very badly behaved and couldn't be trusted
to run in the same process. I have since stopped using this code and
wouldn't have time to get it working again, but it did open and use
shared memory quite nicely on linux.)

Also I found this web site very useful:
http://www.ecst.csuchico.edu/~beej/guide/ipc/

Gael Varoquaux wrote:
> On Thu, Feb 05, 2009 at 05:23:32PM -0600, Robert Kern wrote:
>> BTW, Philip Semanchuk, the maintainer of the aforementioned shm
>> module, contacted Sturla and myself offlist to point out two, more
>> up-to-date, modules which provide named shared memory on UNIX systems:
> 
>>   http://semanchuk.com/philip/sysv_ipc/
>>   http://semanchuk.com/philip/posix_ipc/
> 
> Interesting. I wonder how to use these. I would really like to see shared
> memory in numpy by itself at some point. I did not look at the code as it
> is GPL, from what I see.
> 
> The core idea, from what I understand, would be to use the POSIX shm_open
> call to expose some named shared to numpy using eg from_buffer. Or can we
> simply make it point to the pointer of an existing array using shmat, if
> is is contiguous? That would avoid a copy (if contiguous).
> 
> Finally, to make sure share memory works with multiprocessing, we would
> have to override pickling so that the pickling and unpicking are done
> simply by storing the name of the shared memory object or retrieving it.
> This is risky, because actual persistence would be destroyed.
> 
> Under Window we would use CreateSharedMemory to perform the same trick
> using CreateFileMapping and MapViewOfFile?
> 
> Sounds fun.
> 
> Gaël
> _______________________________________________
> SciPy-user mailing list
> SciPy-user@scipy.org
> http://projects.scipy.org/mailman/listinfo/scipy-user



More information about the SciPy-user mailing list