[IPython-dev] Discussion on Bruce Eckel's blog
Fri Mar 21 15:25:41 CDT 2008
> Is he trying to discuss the issues of parallelism, threading, co-routine
> abstractions, inter process-communication? In the same brief blurb he
> discusses and mixes all four. Makes mention of the GIL and how java
> multithreading through co-routines (stackless???) are... what? I got
> confused even parsing the words. none of these issues are related at
> the level of the post.
> Stack frames, threads, processes, IPC and netowrking are, clearly,
> related but wildly different subjects... not to mention that none of
> these have anything to do with solving parallel computing problems other
> than we might want to use multiple computers and electricity...
I completely agree with this evaluation. Parallel and distributed
computing is a very complex and rich field and throwing all of these
things together really confuses the problems and solutions.
> And IPython1? Sure, there's a notion of a redesign to embrace actual
> parallel computation issues, and even a couplea examples of how one
> might use asynchronous models to implement map-reduce (or scatter-gather
> or whatever)
> But the IPython1 effort is mostly about embracing the asynchronous model
> (read Twisted) with a sugary sweet layer of IPython to make it a bit
> less painful. Unless I missed a bunch of documentation somewhere
> (certainly possible given the state of the documentation), this hardly
> rises to: "a powerful architecture for parallel computing,"
So, I must confess that this is language that I am responsible for, so
it is not something that Bruce can take the blame on. But, I would
argue that the statement is true. At the same time, I admit that we
(mostly myself) have done a really bad job of laying out why this is
the case. This is evidenced by the fact that Parallel Processing and
Processing are being spoken of as the leading solutions in this area -
even though IPython1 is more mature and technically superior. I
should say though that I really like both Parallel Processing and
Processing - I just think we are trying to accomplish something much
more ambitious with IPython1 and are having good successes with our
> IPython1 has a notion
> that there should be an architecture and even some early examples... but
> there's no new model, no power other than doing stuff correctly with
> modern tools.
I like this thinking, even though I don't fully agree. The ironic
thing is that "doing stuff correctly with modern tools" is extremely
difficult and uncommon when it comes to parallel and distributed
computing. Where I disagree slightly is that I think you do get
something really powerful when you 1) have a well designed
architecture and 2) do things correctly with modern tools.
More information about the IPython-dev