[IPython-dev] Moving IPython.kernel.core

Barry Wark barrywark@gmail....
Thu Jun 26 22:38:25 CDT 2008


Brian and I thought it would be good to bring this discussion to
everyone's attention, separate from the frontend plans. Below is the
entire thread, but I've copied the relevant bits just here:

>> It seems wrong that I.kernel.core is a
>> subpackage of I.kernel. I know that it's there to isolate the ipython1
>> stuff from ipython0 stuff, but before too many people start writing
>> code using I.kernel.core, is it worth discussing if there's a better
>> spot for it in the IPython tree?
>
> Yes, probably.  I had originally thoughts about moving it to
> IPython.core.  But the problem with that is I am afraid that it
> suggests that it is a complete and working core.  My plan originally
> was thus:
>
> 1.  Move the old core IPython.*.py -> IPython.core.*.py
>
> 2.  Refactor that stuff until it looks more like IPython.kernel.core
>
> 3.  At that point, get rid of IPython.kernel.core
>
> But maybe the better approach is:
>
> 1.  Just move IPython.kernel.core -> IPython.core
>
> 2.  Also move IPython.*.py -< IPython.core
>
> 3.  Refactor/combine the two inplace
>
> What do you think?  This probably needs more disucssion in a separate
> thread on the list.

Since I don't have any significant code that depends on ipython0, I'd vote for
1. Move IPython.kernel.core -> IPython.core
2. Move IPython.*.py -> IPython.old_core
3. Deprecate IPython.old_core as soon as IPython.core is capable of
replacing ipython0

I'm sure others who have legacy code that depends on ipython0 will
have an opinion...

Barry

On Thu, Jun 26, 2008 at 8:31 PM, Brian Granger <ellisonbg.net@gmail.com> wrote:
> On Thu, Jun 26, 2008 at 9:18 PM, Barry Wark <barrywark@gmail.com> wrote:
>> On Thu, Jun 26, 2008 at 4:03 PM, Brian Granger <ellisonbg.net@gmail.com> wrote:
>>> Barry,
>>>
>>>>> I think that we could have a single base class that implements the
>>>>> actual frontend logic and then have multiple subclasses that handle
>>>>> the calls to the various types of backends (kernel.core or
>>>>> kernel.engineservice).  I need to look more at how the frontend is
>>>>> implemented, but Barry, do you think this would work.
>>>>
>>>> Yes, I think this would work. I thought zope.interface was pure
>>>> python. Since we can't require zope.interface in the stripped-down
>>>> ipython, I now think option 1 is the best approach. I'll work on that.
>>>
>>> Great, I think that will be best, even though it has its own downsides.
>>
>> We're in agreement then. I'll try option 1 and see how that shakes out.
>>
>>>
>>>>> Also, the Synchronous deferred does not present the exact same
>>>>> interface as a true deferred or make the same promises about when
>>>>> things will happen.
>>>>
>>>> What is your understanding of the differences?
>>>
>>> For Deferreds, I think the interface is not a complete specification
>>> of the objects behavior.  For Deferreds, there are subtle cases that
>>> arise when you chain them together.  The behavior exhibited in complex
>>> Deferred chaining _can_ depend on whether or a given Deferred has
>>> fired or not when the chaining is setup.  We have in various places
>>> that handles these odd edge cases.  Also, we sometimes access the
>>> semi-private attributes of Deferreds.  Another thing is that if you
>>> have multiple unfired Deferreds, the order in which they will fire is
>>> not deterministic.  For the synchronous deferreds, I think the
>>> ordering has to be deterministic.  But in my mind, this constitues a
>>> different behavior and thus "interface" in the board sense of the
>>> word.
>>>
>>> We might be able to get the synchronous deferreds to work, but I am a
>>> little hesitant to go down that route simply because all of these
>>> things are _super_ subtle and nearly impossible to debug.
>>
>> I see the issues. Too bad; the Twisted universe seems to want to
>> swallow its victims whole. I should have really seen this coming--I've
>> worked on a Cocoa framework that uses OS X's NSOperationQueue to
>> emulate the Twisted reactor/Deferred system. You're exactly right that
>> the Deferred behavior is _very_ complicated when you dig into it. Oh,
>> well. As long as we're willing to skip the engineservice layer for
>> synchronous interface to core, it's not an issue and we can forget
>> about Synchronous Deferreds.
>>
>>>
>>>> Thus, I my mind, the new IEngineCore
>>>>> implementation _couldn't_ possible really implement the interface.  In
>>>>> fact, if I remember correctly, we have a test that can be run on any
>>>>> IEngineCore implementer.  It actually calls all the methods of the
>>>>> interface and checks:
>>>>>
>>>>> self.assert_(isinstance(return_value, t.i.Deferred)
>>>>>
>>>>> A fake deferred won't pass such tests.
>>>>
>>>> I don't think this is an appropriate test then. The interface should
>>>> specify the behavior of the returned object, not its implementation.
>>>> If the Synchronous Deferred behaves the same as a t.i.defer.Deferred,
>>>> then it should pass the test. The fact that it may not behave the same
>>>> is a bigger issue.
>>>
>>> True, I could write a better test for this.  The current test is only
>>> a super weak test behavior wise, but given the fact that there are no
>>> other objects that truly act like a Deferred, the test suffices.
>>>
>>>>>
>>>>> So, in summary, I think more discussion is needed before commiting to
>>>>> #2.  I am crazy busy, but I will try to have a look at IFrontEnd
>>>>> tonight.
>>>>
>>>>>
>>>>>
>>>>>> Since the IEngine* interface is much more stable than the IFrontEnd
>>>>>> interface, I propose going with solution #2. Although it adds an
>>>>>> additional python file as external dependency, it seems conceptually
>>>>>> cleaner (all frontends go through the engine interface) and allows
>>>>>> frontends to decide between synchronous and asynchronous behavior
>>>>>> without any code changes.
>>>>>
>>>>> I think it is conceptually convoluted because the only significant way
>>>>> that IEngireCore different from the underlying core is that its
>>>>> methods return deferreds (and Failures).  Thus to use IEngineCore, but
>>>>> then try to get rid of the (true) Deferreds by using fake ones seems
>>>>> like double work (why not just call the class whose methods don't
>>>>> return deferreds in the first place).
>>>>
>>>> It feels weird that there are two ways to interact with ipython—engine
>>>> and core directly. It seemed that making a common interface was
>>>> cleaner in that it allowed users of that interface to be able to use
>>>> engine or core without modification of the frontend. Since that's not
>>>> really an option anymore (because of zope.interface), I'm happy to
>>>> give up that opinion and move on.
>>>
>>> I do sort of agree with this, but I think having two interfaces is
>>> appropriate because they have vastly different behaviors (synchronous
>>> vs asynchronous).
>>
>> Fair enough. I'm convinced. I had it wrong in my head-- I.kernel.core
>> is still the 'real' IPython. I.kernel is just a Twised wrapper around
>> that core. Naturally anything that doesn't want what Twisted has to
>> offer should talk directly with I.kernel.core and the price they pay
>> is that they loose all the goodies that come with the I.kernel layer.
>> Now that I just wrote this out, it seems wrong that I.kernel.core is a
>> subpackage of I.kernel. I know that it's there to isolate the ipython1
>> stuff from ipython0 stuff, but before too many people start writing
>> code using I.kernel.core, is it worth discussing if there's a better
>> spot for it in the IPython tree?
>
> Yes, probably.  I had originally thoughts about moving it to
> IPython.core.  But the problem with that is I am afraid that it
> suggests that it is a complete and working core.  My plan originally
> was thus:
>
> 1.  Move the old core IPython.*.py -> IPython.core.*.py
>
> 2.  Refactor that stuff until it looks more like IPython.kernel.core
>
> 3.  At that point, get rid of IPython.kernel.core
>
> But maybe the better approach is:
>
> 1.  Just move IPython.kernel.core -> IPython.core
>
> 2.  Also move IPython.*.py -< IPython.core
>
> 3.  Refactor/combine the two inplace
>
> What do you think?  This probably needs more disucssion in a separate
> thread on the list.
>
> Cheers,
>
> Brian
>
>>>
>>>>> Actually, I think we are in agreement that tab completion (the part
>>>>> that actually looks up things in the users namespace) needs to be done
>>>>> in the engine/core through its complete method.  There is just no way
>>>>> it is reasonable to mirror the user_ns in the frontend for the reasons
>>>>> you mention.  Sorry if we have said anything confusion on this front.
>>>>> So I guess the complete method of the frontend should just call the
>>>>> complete method of the engine/core?
>>>>
>>>> Yes. I agree. frontend.complete() calls engine/core.complete().
>>>
>>> Cool,
>>>
>>> Cheers,
>>>
>>> Brian
>>>
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Brian
>>>>>
>>>>>> Barry
>>>>>>
>>>>>> [1] Removing the Twisted dependence completely will require moving the
>>>>>> IEngine* interfaces to a separate module from the implementations. I
>>>>>> propose moving them to IPython.kernel.engineinterface.py.
>>>>>> _______________________________________________
>>>>>> IPython-dev mailing list
>>>>>> IPython-dev@scipy.org
>>>>>> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>>>>>>
>>>>>
>>>>
>>>
>>
>


More information about the IPython-dev mailing list