[IPython-dev] Re: ipython improvements
Jeffery D. Collins
jcollins at boulder.net
Wed Oct 1 16:53:22 CDT 2003
Fernando Perez wrote:
> Jeffery D. Collins wrote:
>> Hey Fernando,
>> I've been thinking of a few improvements for IPython and thought I
>> would share a few with you.
> I'mm cc-ing the ipyhton-dev list here so the discussion gets archived,
> and also b/c others may also have useful contributions to make.
>> 1. match on the contents of __all__ if present in a module
>> The problem I'm attempting to address is the large number of module
>> attributes when that module imports lots of other objects. One
>> example is: from Numeric import *. When completing on names in the
>> given module, the names the module defines are lost among the imported
>> names. We need some way to complete only those names that the module
>> actually defines. Ideally, there would exist a parameter that would
>> contain this information, but (IIRC) there is none. However, we could
>> use the user-definable __all__ attribute to determine the attribute
>> matches. The disadvantage is that it must be created by the user.
>> I have implemented this feature. If the __all__ attribute is present,
>> those names are used for completion; otherwise, it defaults to the
>> original behavior.
> Great! Is this implemented in the patches you sent me? This is
> something which has irked me always, but I never really looked into
> fixing it. I think your approach of using __all__ as a marker of what
> to complete on is the right one. If it was not in your original
> patches, perhaps you can unify it and send it my way. I still haven't
> looked at the others, so I could work on the whole thin in one pass.
The patch is on its way...
>> 2. configuration
>> What I need at the moment (if it doesn't exist) is a 'from x import y
>> as z' mechanism. As an alternative, I could just import x.y and
>> assign that to z. This should be possible, correct?
> Yes. But you can also simply 'execute from x import y as z' in the
> config file, or even 'execfile my_extra_configthings.py'. The 'execute'
> option is for single lines of plain python code you want to run at
> startup, and the 'execfile' option gives you full freedom to have an
> arbitrarily complex startup file which is regular python code.
Thanks. I really should read the manual more carefully about this:)
>> We have discussed configuration briefly in the past. You noted at the
>> time that it would have been better to use python itself instead of
>> specialized configuration files. How difficult would it be to make
>> such changes?
> Difficult, not really. But time-consuming, quite. The big problem is
> the startup ipmaker routine, which is a single, very long function which
> carefully tiptoes through bringing ipython up with zero modularity.
> This stuff should probably all go into the constructor, or perhaps
> partly into __init__ and partly into an explicitly callable initialize()
> method, I'm not sure. I keep the 2nd option open, because it might be
> useful to have the constructor be reasonably simple, with an initializer
> which must be called before running providing fine-tuning, for the
> purposes of embedders.
> In cleaning up ipmaker, one of the first things to go would be the
> baroque config system currently in place. The only slightly delicate
> problem is how to handle recursive inclusions, so that one config file
> can include another, and in the end have it all work transparently for
> the user. That requirement is key, because it is what makes profiles in
> ipython a powerful customization tool. Initially I thought it couldn't
> be solved if the config files were real python code, but that was just a
> sign of my inexperience with the language. With a bit of care that's not
> a problem, and removing the current config manager would vastly simplify
> a lot of code.
I agree that this is quite feasible with python. There may already
exist an implementation somewhere on which to model it.
>> I guess that leads to the bigger question of how to reformulate
>> IPython given the lessons-learned. I am pretty satisfied with the
>> current behavior, use it all of the time and in fact am now unable to
>> use the default interactive environment! Just looking quickly among
>> the modules, has python been sufficiently patched to allow the removal
>> of FlexCompleter? Is DPyGetOpt really needed? Python already has a
>> fairly complete version of getopt, now that optparse has made it into
>> the distribution. For these, I realize these changes may introduce be
>> backwards compatibility problems that would need to be resolved. In
>> all fairness, these are pretty minor issues. I'll get back with you
>> when I've had a chance to look through the code. BTW, which is the
>> appropriate forum for discussing IPython development? scipy-dev/user?
>> Is there a thread on this; if not, should one be started? :)
> Well, I just started it on ipython-dev with this reply to your mail :)
> If you're not subscribed to the list, don't be afraid, it's very low
> Your other comments are on the nose, and basically mimic what I've been
> thinking lately. I'm leaning strongly towards making 2.3 a requirement
> for ipython when the internal cleanup starts. The reasons:
> - DPyGetOpt: replace it with getoptparse.
> - Logger: replace it with the standard logging module (I think).
> - FlexCompleter: that's the new rlcompleter.py (it was my patch :)
> - Itpl: remove altogether, we can do without.
> These changes would mean yanking a fair bit of code out of ipython,
> which is always a good thing in my book. Since this rewrite will likely
> take some time, probably by the moment it's production-ready, most
> people won't see the 2.3 requirement as a big burden. And I'd still
> keep the 0.5x branch around for critical fixes for users of py2.1.
> If the 2.3 requirement is a big problem for many people, one could
> always package getoptparse and logger from 2.3 along with ipython, I
> suppose. But I'll worry about that when the time comes.
I agree with the 2.3 requirement here. I only occasionally refer to 2.2
for debugging purposes when some odd or unexpected behavior occurs. If
there is a need to backport standard packages, a subdirectory could
contain these modules.
> Basically the cleanup would entail:
> - Cleaning ipmaker, with removing the current config handler as part of
> - Making Magic.py NOT be a mixin, but rather an attribute of the ipython
> shell itself (self.MagicSystem = Magic() or somesuch).
> - The 4 module removals mentioned above (requiring 2.3)
> Please also take a look at the new_design document, which is where I try
> to keep notes on this whole problem so everyone has access to them. I
> may even port this thread back to that document later.
Where can I find this document? SF (seems to be down at the moment)?
Jeffery Collins (http://www.boulder.net/~jcollins)
More information about the IPython-dev