[SciPy-user] General parallel processing question

Brian Granger ellisonbg.net at gmail.com
Mon Dec 4 14:49:07 CST 2006


> I just want to clear up any misconception about Pyro - there are several
> ways to avoid the issue Brian mentions. The easiest are "one way" calls.
> Another approach is to use multi-threading. Anyhow, I find Pyro to be a
> quick and easy way to get started on IPC.

There is no doubt that Pyro is a quick and easy approach.  I like it
very much for that.  The workarounds you mentions definitely work.  I
wasn't familiar with the "one way" calls but after reading about them,
they seems like a reasonable thing that doesn't require threads.

> [Note: Pyro's approach is apparently inflammatory to some of the Twisted
> developers, I think because they're (rightfully) concerned about
> security. However, if you're running on a single, (possibly multi-core)
> machine or a trusted LAN, the security issue is a non-issue.]

Yes, they strongly object to the usage of pickle based RPCs.  I
understand their security concerns, but I think there *are* times
(such as the usage case you mention) where those concerns are not a
show stopper for pickle based RPCs.  We definitely use pickle to send
around arbitrary objects in ipython and this probably won't change.

Even though I like Pyro in many ways, the main objection I have it
that there is no clean abstraction of asynchronous events.  At the
root level Pyro is synchronous and threading/one way calls only make
things look asynchronous.  For more complicated distributed systems
that are really asynchronous  in nature, I think Pyro's simplicity is
lost because of this.  At that point, it is easier just to bite the
bullet and use the clean abstractions that Twisted provides.


Brian


> -Andrew
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://projects.scipy.org/mailman/listinfo/scipy-user
>


More information about the SciPy-user mailing list