[SciPy-User] Speeding things up - how to use more than one computer core

Neal Becker ndbecker2@gmail....
Mon Apr 8 06:35:30 CDT 2013


Gael Varoquaux wrote:

> On Sun, Apr 07, 2013 at 01:11:09PM -0400, Neal Becker wrote:
>> > Regarding the difference between processes and threads:
>> ...
>> > On the other hand, sharing data between threads is much cheaper than
>> > between processes.
> 
>> I have to take issue with this statement.  Sharing data could suffer no
>> overhead at all, if you use shared memory for example.
> 
> How do you use shared memory between processes?
> 
> There are solutions, but hardly any are easy to use. I'd even say that
> most are very challenging, and the easiest option is to rely on memapped
> arrays, but even that is a bit technical, and will clearly introduce
> overhead.
> 
> Gaël

Why do you think memmaped arrays would introduce overhead?  The only overhead 
should be if you have to add some sort of synchronization between writers and 
readers (e.g., semaphores).  The actual data access is as fast as any other 
memory access.



More information about the SciPy-User mailing list