[SciPy-user] shared memory machines

Brian Granger ellisonbg.net@gmail....
Thu Feb 5 22:20:04 CST 2009


Thanks!

On Thu, Feb 5, 2009 at 8:13 PM, Andrew Straw <strawman@astraw.com> wrote:
> FWIW, I wrote some BSD licensed Pyrex code that does some shared memory
> stuff. I wouldn't attempt to resurrect the complete working module, but
> cut and paste at will:
>
> http://code.astraw.com/projects/motmot/browser/trunk/pycamiface/src/_cam_iface_shm.pyx?rev=328
>
> (This was from a wrapper of a camera driver that used shared memory
> since the camera driver was very badly behaved and couldn't be trusted
> to run in the same process. I have since stopped using this code and
> wouldn't have time to get it working again, but it did open and use
> shared memory quite nicely on linux.)
>
> Also I found this web site very useful:
> http://www.ecst.csuchico.edu/~beej/guide/ipc/
>
> Gael Varoquaux wrote:
>> On Thu, Feb 05, 2009 at 05:23:32PM -0600, Robert Kern wrote:
>>> BTW, Philip Semanchuk, the maintainer of the aforementioned shm
>>> module, contacted Sturla and myself offlist to point out two, more
>>> up-to-date, modules which provide named shared memory on UNIX systems:
>>
>>>   http://semanchuk.com/philip/sysv_ipc/
>>>   http://semanchuk.com/philip/posix_ipc/
>>
>> Interesting. I wonder how to use these. I would really like to see shared
>> memory in numpy by itself at some point. I did not look at the code as it
>> is GPL, from what I see.
>>
>> The core idea, from what I understand, would be to use the POSIX shm_open
>> call to expose some named shared to numpy using eg from_buffer. Or can we
>> simply make it point to the pointer of an existing array using shmat, if
>> is is contiguous? That would avoid a copy (if contiguous).
>>
>> Finally, to make sure share memory works with multiprocessing, we would
>> have to override pickling so that the pickling and unpicking are done
>> simply by storing the name of the shared memory object or retrieving it.
>> This is risky, because actual persistence would be destroyed.
>>
>> Under Window we would use CreateSharedMemory to perform the same trick
>> using CreateFileMapping and MapViewOfFile?
>>
>> Sounds fun.
>>
>> Gaël
>> _______________________________________________
>> SciPy-user mailing list
>> SciPy-user@scipy.org
>> http://projects.scipy.org/mailman/listinfo/scipy-user
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user@scipy.org
> http://projects.scipy.org/mailman/listinfo/scipy-user
>


More information about the SciPy-user mailing list