[IPython-user] pysh variable substituion and scope

[Krishna Mohan Gundu]

Hi Fernando,

> Please always attach patches, never inline them.  Python is finicky
> enough about whitespace that it's /very/ easy to end up with a subtly
> broken patch due to indentation issues, after being mangled by the
> mail client's idea of linebreaks.  In your case, I couldn't even get
> the patch to apply at all ('malformed patch') by pasting it.  It's
> short enough that I just did it by hand, but that's not viable in for
> bigger ones.  Some info on the matter:

Sorry for the trouble. I cannot however, for now, provide a diff from
svn as I have other modification I submitted earlier (although
independent).

> The reason why this kind of hack is not a good idea is that it doesn't
> work with more complex cases than your original simple test:
>
>
> longs[~]|2> def foo(x):
>         |.>     def bar():
>         |.>         echo "$x"
>         |.>     bar()
>         |.>
> longs[~]|3> foo(3)
>
> Granted, the example above looks a bit contrived, but nested scopes
> are part of the language:

This is a valid example where the code breaks. I expected this and
probably should have waited submitting a patch until I made amends.

> Lexical scoping is a bit tricky to deal with, because most tools (even
> in the language itself, things like eval) can only accept at most a
> globals/locals pair of namespaces for evaluation.  But the language
> itself does (as of 2.1) allow for nested scopes.

I am beginning to notice that. It sounds strange to me that the
interface provided to evaluate expressions does not coinicide with the
default interpreters behavior.

> Since you can't know how many frames can be in between the caller and
> the declaration of the variable, a sys._getframe() hack will never do
> this correctly for all cases.  One possible solution might be (I'm not
> 100% sure) to properly use, at interpolation time, the func_closure
> generated by the language.  This might let you really match the
> language's rules for scoping, explained here:

Unfortunately func_closure has info about only free variables (I still
have to get a hang of what free variables mean)

> Right now I'm not really sure how to do that correctly, and it would
> have to be done (I think) in Itpl's interpolation code.  I thought of
> walking the stack (the f_back attributes let you do this easily), but
> you'd need to be sure to stop before hitting ipython's own frames, to
> avoid ipython's internal variables from being interpolated out.  It's
> a tricky problem, and given that the language's own machinery already
> does this correctly, I think it's saner to simply let Python deal with
> it.

We cannot do frame backtracking in Itpl since the IPSHELLs execution
stack, so to say, is intervened by call to ipsystem(), which is
executed in a different namespace. Which means Itpl needs to know more
information than it ideally should. I think the right place for it is
ipsystem(). Currently I used a signature to identify the global
namespace of IPSHELL by setting the variable __sig__ in __builtins__.
I used the value 0xa005 because I vaguely remember that's the
signature for boot-sector. One can then backtrack until one exits the
global namespace identified by this signature. In case of embedding
one needs to set/verify signature in appropriate namespace. I dont
know if this solves the problem in general (not likely), since I am
simply merging/overriding the locals from every frame. I will read the
scoping rules of various code blocks and modify it appropriately in
future.

> I'd suggest, if you find that your own code is proving cumbersome to
> write for other reasons, that you try to design a few good utility
> objects that expose the functionality you need in the cleanest
> possible fashion.  That will be, in the long term, a far more robust
> and reusable solution than on-the-fly language rewriting by an
> interactive shell.

All I want really is the ability to run shell commands with variable
substituion from anywhere in the python code. I am not thinking big
now.

Example:
==test.ipy==
x = 1
y = 2
z = 3
w = 4
def foo(x):
   def bar(y):
      def foobar(z):
         echo "$x $y $z $w"
      foobar(1)
   bar(2)

foo(3)
==test.ipy==
prints "3 2 1 4"

I dont have any examples right now with more complex expressions
involving classes, lambdas and the like.

cheers,
Krishna.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: iplib.py.patch
Type: text/x-patch
Size: 2560 bytes
Desc: not available
Url : http://projects.scipy.org/pipermail/ipython-user/attachments/20060604/8bc6b86c/attachment.bin