[Numpy-discussion] passing arrays between processes

Bryan Cole bryan@cole.uklinux....
Sun Jun 14 14:31:49 CDT 2009


I'm starting work on an application involving cpu-intensive data
processing using a quad-core PC. I've not worked with multi-core systems
previously and I'm wondering what is the best way to utilise the
hardware when working with numpy arrays. I think I'm going to use the
multiprocessing package, but what's the best way to pass arrays between
processes?

I'm unsure of the relative merits of pipes vs shared mem. Unfortunately,
I don't have access to the quad-core machine to benchmark stuff right
now. Any advice would be appreciated. 

In case it's relevant: the data takes the form of a stream of
numpy.double arrays with sizes in the range 2000 to 10000. 

cheers,
Bryan




More information about the Numpy-discussion mailing list