[IPython-user] Notes on using ipython as a system shell ('pysh')

Fernando Perez Fernando.Perez at colorado.edu
Tue Apr 13 15:39:38 CDT 2004

Hi all,

I'll reply here, to collect both what Prabhu and Daniel have said.

Daniel 'Dang' Griffith wrote:
>    -------Original Message-------
>    > From: Prabhu Ramachandran <prabhu_r at hathway.com>
>    > Subject: Re: [IPython-user] Notes on using ipython as a system shell ('pysh')
>    > Sent: 13 Apr 2004 13:48:57
>    >
>    >  >>>>> "FP" == Fernando Perez <Fernando.Perez at colorado.edu> writes:
>    >  
>    >      FP> 4.  It should also be possible to expand python
>    >      FP>     variables/commands in the
>    >      FP> middle of system commands.  I thihk this will make it
>    >      FP> necessary to use $var for name expansions:
>    >  
>    >      >>> var='hello' # var is a Python variable print var
>    >      FP> hello # This is the result of a Python print command
>    >      >>> echo $var
>    >      FP> hello # This calls the echo command, expanding 'var'.
>    >  
>    >  Everything else sounds nice.  I am not comfortable with '$var'.  What
>    >  I would really like to see is that variables are just like any other
>    >  Python variable.  Its just that shell commands are part of the
>    >  vocabulary.  Or is that hard to implement?  For example:
>    >  
>    >  >> var = 'hello'
>    >  >> echo var + ' world!'
>    >  hello world!
> Without a $prefix, I don't see how ipython could tell the difference 
> between a token the user types in that is intended to be passed 
> as a literal command argument and one that is to be interpolated 
> as a python variable.  Most of the shells have some kind of quoting 
> mechanism to indicate whether tokens should be evaluated or left 
> as literals.  Maybe a similar mechanism is needed here.
> I'm not adverse to a $prefix.  Other shells require something to distinguish
> literals from variables.  If ipython is a shell, I don't see why it shouldn't
> be able to do the same.  Python itself doesn't require the prefix 
> because it has a highly constrained grammar.  When you're working
> in a system shell, every command could have its own syntax.

I agree with Daniel here, in that I simply don't see a clean way to handle the 
two namespaces (filesystem and python variables) without an explicit escaping 
mechanism.  I'd love to hear a bit more detail from Prabhu, if you see a way 
out.  The problem is that we want to be able to set a variable:

 >> var='FIXME'

and then issue a shell command which uses the _value_ of the variable:

 >> grep $var *.c

Since shell commands are interpreted literally, how would we do the above 
without some escaping mechanism?  I don't want to allow:

 >> grep var *.c

to interpolate the value of var, for several reasons:

1.  It would be very surprising.  I think anyone using a shell would expect 
the above to simply produce  a search for the string 'var' in all files with a 
.c extension, and we should keep that behavior.

2.  The implementation would in fact be much trickier.  I can scan lines which 
are going to the shell for $ and expand those using Itpl, the string 
interpolation library written by Ka-Ping Yee, whic IPython already uses 
internally.  The nice thing is that:

   a.  Itpl already uses $ as its interpolation symbol, so the implementation 
is trivial.

   b.  Users are accustomed to $var being a way to expand a variable in _all_ 
shells.  Even though the syntax for setting values varies from shell to shell, 
the expansion syntax is universal.  I think this is important, because pysh 
should offer as much out-of-the-box compatibility with other shells as 
possible.  This, both to lower the barrier of entry and to make life easier 
for pysh users when they are on non-pysh systems.

For the above reasons, I honestly fail to see how to do without an explicit 
interpolation symbol for expanding python variables into commands meant to go 
to the system.  Obviously, any code which is normal python would obey the 
regular python syntax rules.

> To make it appear "pythonic", maybe variables could be wrapped with 
> the dictionary variable interpolation "%(myvar)s" syntax, using the 
> shell's globals() as the dictionary.  I haven't thought this through
> to its conclusion, and I'm not really advocating it, but it is pythonic, 
> though also a bit perlish.
> E.g.,
>    >  >> var = 'hello'
>    >  >> echo %(var)s + ' world!'
>    >  hello world!

-1.  A shell must value economy of typing above all.  I've always hated %()s 
even in python programs for being too much to type.  I also want to keep the 
user-visible behavior of pysh as close to that of known shells as possible.

> Although since the command arguments are always strings, I
> don't suppose the trailing "s" in "%(var)s" is needed.
> ...
>    >  Also can pipes be handled?
>    >  
>    >  >> ps uax | grep ipython | sort > /tmp/ps

Something like the above works even now, as long as you start with !.  It's 
just a call to os.system(), and it would remain so.  The difference is that 
pysh would keep stdout/err trapped always into _ and _e for your convenience, 
so its calling mechanism would actually be like what @sx does today, instead 
of what ! does.

>    >      FP> - it's ok for shell builtins (in this case this includes the
>    >      FP>   python language)
>    >      FP> to override system commands on the path.  See tcsh's 'time' vs
>    >      FP> '/usr/bin/time'.  This settles the 'print' issue and related.
> +1.  I mentioned the print conflict when this came up, but I completely 
> agree that the shell should supercede a command in the path, if only
> because there are other ways to invoke the command unambiguously.


Thanks to all for the feedback.

I should reiterate that, regardless of the fact that I'm liking this idea more 
and more, I'm _not_ going to let myself get sucked into doing it :) 
Implementing something like this is going to be quite time-consuming, given 
the state of affairs of ipython's internals.



More information about the IPython-user mailing list