[SciPy-user] General parallel processing question
strawman at astraw.com
Fri Nov 17 12:39:17 CST 2006
Brian Granger wrote:
> There is one critical point though that in my mind makes non-blocking
> sockets absolutely necessary for this type of thing (Twisted uses
> non-blocking sockets I don't think Pyro does). If you use blocking
> sockets a CPU bound activity in one process will *completely block*
> the execution path in the other process. Because of this if you use
> blocking sockets your system will not experience any performance
> benefits even though it appears to be running in parallel.
I just want to clear up any misconception about Pyro - there are several
ways to avoid the issue Brian mentions. The easiest are "one way" calls.
Another approach is to use multi-threading. Anyhow, I find Pyro to be a
quick and easy way to get started on IPC.
Pyro's approach is apparently inflammatory to some of the Twisted
developers, I think because they're (rightfully) concerned about
security. However, if you're running on a single, (possibly multi-core)
machine or a trusted LAN, the security issue is a non-issue.
More information about the SciPy-user