[Numpy-discussion] Annoyance of memap rraywithmultiprocessing.Pool.applay_async

Nadav Horesh nadavh@visionsense....
Mon Apr 5 03:08:08 CDT 2010

Is there a way to use memory mapped files as if they were shared memory? I made an application in which some (very often non contiguous) parts of a memmap array are processed by different processors. However I might use shared memory array instead. I wonder, since both types share common properties, if there a way to interchange then transparently.


-----Original Message-----
From: numpy-discussion-bounces@scipy.org on behalf of Robert Kern
Sent: Sun 04-Apr-10 18:45
To: Discussion of Numerical Python
Subject: Re: [Numpy-discussion] Annoyance of memap rraywithmultiprocessing.Pool.applay_async
On Sat, Apr 3, 2010 at 22:35, Nadav Horesh <nadavh@visionsense.com> wrote:
> Got it, thank you.
> But why, nevertheless, the results are correct although the pickling is impossible?

Rather, I meant that they don't pickle correctly. They use ndarray's
pickling, which will copy the data, and then reconstruct an ndarray on
the other side and just change the type to memmap without actually
memory-mapping the file. Thus you have a __del__ method referring to
attributes that haven't been set up.

Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco
NumPy-Discussion mailing list

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/ms-tnef
Size: 3824 bytes
Desc: not available
Url : http://mail.scipy.org/pipermail/numpy-discussion/attachments/20100405/e21fa263/attachment.bin 

More information about the NumPy-Discussion mailing list