[IPython-user] IPython.fronend progress
Sat Jun 21 16:58:38 CDT 2008
On Fri, Jun 20, 2008 at 12:25 AM, Fernando Perez <firstname.lastname@example.org> wrote:
> Hey Barry,
> sorry for the delay in replying, I've had a semi-drafted reply for days...
No worries. I'm on the slow track with things anyways...
> On Sun, Jun 15, 2008 at 3:32 PM, Barry Wark <email@example.com> wrote:
>> The purpose of IPython.frontend is to provide a base class
>> (frontend.frontendbase.FrontEndBase) for frontend writers to use in
>> developing GUI frontends for IPython. FrontendBase provides all of the
>> basic functionality required by the front end (testing for block
>> completeness, managing history, executing blocks on an
>> IPython.kernel.engineservice.IEngineBasic-implementing engine, and
>> rendering results and errors asynchronously (via Twisted deferreds)).
>> GUI frontends will want to subclass FrontEndBase and override the
>> render methods (render_result and render_error) to render results and
>> errors respectively. Each execute request can be assigned a "blockID"
>> to allow the frontend to identify the block corresponding to the
>> result/error. The frontend can assign this blockID or a UUID will be
>> assigned automatically. Frontends may also override update_prompt to
>> update the input prompt for a block once it's number is known (it may
>> not be known until the engine executes the block if more than one
>> client is using the same engine). The subclass must also decide when
>> to send a block to the engine. For this, FrontEndBase provides
>> is_complete to test whether a block is a complete statement. Once a
>> block is complete, the frontend can send it to the engine by passing
>> it to FrontEndBase's execute method. The result of execute is a
>> twisted.internet.deferred.Deferred result of the execution of that
> I think I mostly agree with your approach so far, but there's a point
> that I'd like to clarify. I just spoke with Brian and he mentioned
> you'd discussed this already at length, so perhaps I'm just
> misunderstanding slightly your angle in the above pargraph.
> It seems to me, from this description, that you intend the
> FrontEndBase class itself to have a twisted core to it and to return
> deferreds everywhere. Am I reading that correctly? I don't think
> this is the direction we want to go in, since in the end there's a
> use case for a non-twisted, pure terminal client that can be installed
> just on top of the python stdlib and nothing more. We have that
> today and I think we can continue to maintain that. (As for building
> full-fledged GUI or network-aware clients, then definitely I agree
> that twisted is our first and foremost dependency, no problem there).
Brian and I have discussed this at some length, but I don't feel that
a perfect solution has presented itself yet. Of course, I'm coming at
this with much less knowlege of the IPython kernel/corearchitecture
and of Twisted than you or Brian, but here's my story.I did try to
architect things so that Twisted-dependent and Twisted-free frontends
would come naturally from a single base. I don't think that this is
the right goal, however. The GUI frontends (those that I assume will
be Twisted-dependent) naturally fit into an asynchronous model AND can
use an IPython.kernel.IEngineBase implementor as their interface to
the interpreter/core. The Twisted-free frontend will have to skip the
IEngineBase layer and go straight to the interpreter/core unless we
rewrite significant parts of Twisted's deferred architecture to
simulate deferred results in a Twisted-free, synchronous environment
or write an IEngineBase provider that skips all of the Twisted stuff
(more on this later). Furthermore, there are many use-cases for GUI
frontends (namely every GUI toolkit) and only a single use case (i.e.
terminal) where being Twisted-free is not a disadvantage. The
advantages of using Twisted and the engine interface for as many use
cases as possible include not having to re-invent the wheel and making
frontends capable of switching to a distributed cluster rather than a
single in-process engine just by chaning the engine that they
communicate with (very cool). Therefore I think the following makes
1. Define an IFrontEnd interface (is it OK to use zope.interface?) [done]
2. Redefine FrontEndBase as TwistedFrontEndBase to indicate that it
depends on Twisted. This class provides IFrontEnd.
3. Write CoreFrontEndBase that implements IFrontEnd, but does not
return deferreds. Instead, I assume it will use an explicit callback
mechanism and will communicate directly with the core, rather than via
an Engine interface. This is instead of writing an IEngineBase
implementation that is Twisted-free.
4. Write the terminal frontend using CoreFrontEndBase. In reality, I
think the terminal frontend will be the only use of CoreFrontEndBase,
but I may just be short-sighted on that. Hopefully someone will be
willing to step up and help on this since I've got my hands pretty
full and I assume this will be the most work of the 4 tasks. Having an
other set of eyes looking at the problem of the architecture of
IFrontEnd would help too.
What do you think?
> I'm right now mostly working on teaching Nose about all of ipython's
> peculiarities regarding testing, so that we can do this refactoring
> you're headed towards on the existing codebase. But I want to make
> sure that we don't stall you out.
Definitely not a problem. I've got way too much on my plate at the
moment so I'm just poking at this here and there. It's a good idea to
start this planning now, however, so that there's infrastructure
inplace to make use of the refactored core when it's available...
More information about the IPython-user