[IPython-dev] Coordinating the XREQ and SUB channels
Thu Jul 15 11:23:57 CDT 2010
I've added a 'flush' method to the KernelManager here:
It works, although there may be a more intelligent way to do it. That being
said, I tried a number of different things, and none of the others worked.
Brian: since the 'flush' method must be called explicitly by clients, this
won't break our model or extra induce latencies for clients that want to
take a more sophisticated approach to SUB channel monitoring.
On Wed, Jul 14, 2010 at 4:21 PM, Evan Patterson <firstname.lastname@example.org>wrote:
> On Wed, Jul 14, 2010 at 2:10 PM, Brian Granger <email@example.com>wrote:
>> On Wed, Jul 14, 2010 at 10:56 AM, Evan Patterson <firstname.lastname@example.org>
>> > Hi guys,
>> > I've been making decent progress at connecting my FrontendWidget to a
>> > KernelManager. I have, however, encountered one fairly serious problem:
>> > since the XREQ and SUB channels of the KernelManager are in separate
>> > threads, there is no guarantee about the order in which signals are
>> > I'm finding that 'execute_reply' signals are frequently emitted *before*
>> > the output signals have been emitted.
>> Yes, that is definitely possible and we really don't have control over
>> it. Part of the difficulty is that the SUB/SUB channel does buffering
>> of stdout/stderr (just like sys.stdout/sys.stderr). While it will
>> make your application logic more difficult, I think this is something
>> fundamental we have to live with. Also, I wouldn't be surprised if
>> the same were true of the regular python shell because of the
>> buffering of stdout.
> I'm not sure it's fair to call this problem fundamental (if we ignore the
> corner case of the threads in the kernel). After all, output and execution
> completion happen in a very predictable order in the kernel; it's only our
> use of multiple frontend-side channel threads that has complicated the
> In a regular same-process shell, this wouldn't be a problem because you
> would simply flush stdout before writing the new prompt. It makes sense to
> be able to request a flush here, I think. A 'flush' in this case would just
> consist of the making the SubChannel thread active, so that its event loop
> would pick up whatever it needs to. I believe calling time.sleep(0) once in
> the XReqChannel before sending an execute reply will be sufficient. The
> latency introduced should be negligible. I'll experiment with this.
>> > It seems to me that we should be enforcing, to the extent that we can
>> > ignoring threads in the kernel for now), the assumption that when
>> > 'execute_reply' is signaled, all output has been signaled. Is this
>> > reasonable?
>> I don't think so. I would write the frontend to allow for arbitrary
>> timing of the execute_reply and the SUB messages. You will have to
>> use the parent_id information to order things properly in the GUI.
>> Does this make sense. I think if we try to impose the timing of the
>> signals, we will end up breaking the model or introducing extra
>> latencies. Let us know if you have questions. I know this will
>> probably be one of the more subtle parts of the frontend logic.
> Yes, this is something that will be quite difficult to get right. For
> frontend implementors who are interested only in console-style interaction,
> it doesn't make sense for them to have worry about this.
>> > Evan
>> Brian E. Granger, Ph.D.
>> Assistant Professor of Physics
>> Cal Poly State University, San Luis Obispo
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the IPython-dev