[IPython-user] pysh variable substituion and scope

Fernando Perez fperez.net at gmail.com
Sun Jun 4 08:41:56 CDT 2006


Hi Krishna,

On 6/4/06, Krishna Mohan Gundu <gkmohan at gmail.com> wrote:

> After long hard fight (~ 8hrs) of reading through python reference
> manuals and going through ipython code (I learnt a lot no regrets), I
> find that this is pretty easy to implement (Actually I only changed
> three lines of code in iplib.py). The basic problem is that in
> interactive mode, for whatever reason, a local namespace is not stored
> and hence whenever the execution is in a code block, a call to eval
> (through Itpl) with only global namespace fails to recognize local
> variables. The fix is to pass the local namespace as well to Itpl.
> This can be done either by saving the local namespace in ip or
> extracting the local namespace from the stack in ipsystem(). Both are
> easy to implement but the latter is simpler. The diff is attached
> inline if anyone is interested.

Please always attach patches, never inline them.  Python is finicky
enough about whitespace that it's /very/ easy to end up with a subtly
broken patch due to indentation issues, after being mangled by the
mail client's idea of linebreaks.  In your case, I couldn't even get
the patch to apply at all ('malformed patch') by pasting it.  It's
short enough that I just did it by hand, but that's not viable in for
bigger ones.  Some info on the matter:

http://projects.scipy.org/ipython/ipython/wiki/DeveloperGuidelines


> I understand that this falls into the territory of supporting
> non-python type code. But I feel if it affords convenience I am all
> for it. My basic complaint against python is that its file handling
> and input/output redirection through pipes is tedious like C (although
> much better than C). I need something that is more shell like. Not
> having this ability in the middle of a function call does not sound
> good to me. Although this is not an efficient method, I am willing to
> lose speed over convenience. Any insight in this regard is welcome.

The reason why this kind of hack is not a good idea is that it doesn't
work with more complex cases than your original simple test:


longs[~]|2> def foo(x):
        |.>     def bar():
        |.>         echo "$x"
        |.>     bar()
        |.>
longs[~]|3> foo(3)
---------------------------------------------------------------------------
exceptions.NameError                                 Traceback (most
recent call last)

/home/fperez/<ipython console>

/home/fperez/<ipython console> in foo(x)

/home/fperez/<ipython console> in bar()

[snip]

/home/fperez/ipython/svn/ipython/trunk/IPython/Itpl.py in _str(self, glob, loc)
    199         app = result.append
    200         for live, chunk in self.chunks:
--> 201             if live: app(str(eval(chunk,glob,loc)))
    202             else: app(chunk)
    203         out = ''.join(result)

/home/fperez/<string>

NameError: name 'x' is not defined

Granted, the example above looks a bit contrived, but nested scopes
are part of the language:

longs[~]|4> def foo(x):
        |.>     def bar():
        |.>         print x
        |.>     bar()
        |.>
longs[~]|5> foo(3)
3


Lexical scoping is a bit tricky to deal with, because most tools (even
in the language itself, things like eval) can only accept at most a
globals/locals pair of namespaces for evaluation.  But the language
itself does (as of 2.1) allow for nested scopes.

Since you can't know how many frames can be in between the caller and
the declaration of the variable, a sys._getframe() hack will never do
this correctly for all cases.  One possible solution might be (I'm not
100% sure) to properly use, at interpolation time, the func_closure
generated by the language.  This might let you really match the
language's rules for scoping, explained here:

http://www.python.org/dev/peps/pep-0227/

Right now I'm not really sure how to do that correctly, and it would
have to be done (I think) in Itpl's interpolation code.  I thought of
walking the stack (the f_back attributes let you do this easily), but
you'd need to be sure to stop before hitting ipython's own frames, to
avoid ipython's internal variables from being interpolated out.  It's
a tricky problem, and given that the language's own machinery already
does this correctly, I think it's saner to simply let Python deal with
it.

I'd suggest, if you find that your own code is proving cumbersome to
write for other reasons, that you try to design a few good utility
objects that expose the functionality you need in the cleanest
possible fashion.  That will be, in the long term, a far more robust
and reusable solution than on-the-fly language rewriting by an
interactive shell.

> @@ -896,7 +896,8 @@
>      def ipsystem(self,arg_s):
>          """Make a system call, using IPython."""
>
> -        self.system(arg_s)
> +        lvars = sys._getframe(0).f_back.f_locals

FYI, this can be done more simply as:

 lvars = sys._getframe(1).f_locals


Cheers,

f




More information about the IPython-user mailing list