[IPython-user] New feature in CVS, testers?

[Fernando Perez]

Hi all,

something had always nagged me about ipython: the asymetry between output 
capture of python computations and that of shell commands.  For shell output, 
you had to use the sc/sx magics, and manipulating these is not always too 
convenient.  I often found myself repeating a command simply because I didn't 
capture it, or wanting to do something to the last lines of output, etc. 
Ville's recent request is a good example of this kind of need.  While I didn't 
want to mess with the interpolation system, I think I found a much cleaner, 
generic solution by other means.  But I'd really like some feedback and 
testing on this, because the changes touch quite a few places.

Here's a quick rundown of some of the things you can do now.  For more 
details, consult the new %sc and %sx docstrings.  The relevant manual section 
says:

Automatic capture of command output

IPython will always store the contents of the standard output and error of the 
last command passed to the system (whether via ! or via an alias) in the 
special variables _o (for stdout) and _e (for stderr). These variables hold 
the resulting strings, but they also provide a special attributes:

.l (or .list): value as list (split on newlines).
.n (or .nlstr): original value (the string itself).
.s (or .spstr): value as whitespace-separated string.

This system is similar to that used by IPython for caching the results of 
Python computations, described in Sec. [sec:cache_input]. It makes it very 
conveninent to quickly perform operations on the result of any system command:

In [11]: ls *py
setup.py*
win32_manual_post_install.py

# the original output:
In [12]: _o
Out[12]: 'setup.py*\nwin32_manual_post_install.py'

# the output as a string with whitespace
In [13]: _o.s
Out[13]: 'setup.py* win32_manual_post_install.py'

# this can be passed to a shell command:
In [14]: !wc -l $_o.s
   146 setup.py
   130 win32_manual_post_install.py
   276 total

Similarly, you can use the list form for more complex manipulations:

In [15]: ls *py
setup.py*
win32_manual_post_install.py

# this output as a list:
In [16]: _o.l
Out[16]: ['setup.py*', 'win32_manual_post_install.py']

# You can loop on the list form with Python code:

In [17]: for f in _o.l:
    ....:      !wc -l $f
    ....:
146 setup.py
130 win32_manual_post_install.py

# Again, _o will hold the last system call:
In [18]: _o
Out[18]: '130 win32_manual_post_install.py'

############################

The whole system is implemented on top of a pair of list/string-like classes, 
which allow these .l, .n and .s access forms uniformly.  All capture 
mechanisms use these, so you should be able to use it efficiently without 
having to remember many details.  Basically it boils down to:

1. All system/alias calls populate _o and _e with stdout/err, with these 
special strings.  This is similar to how all computations populate _ 
automatically.

2. All explicit captures to named strings use these same special strings.

3. All captures to named lists use these special lists.

And regardless of whether you captured to list or to string, you can always 
use: foo.l, foo.s and foo.n to get the list, space-separated or 
newline-separated form.

I would greatly appreciate feedback on whether this is:

1. useful

2. clear, easy to use, intuitive (esp. in terms of uniformity with the rest of 
the environment)

3. stable/bugfree.

Those on CVS can pull it from there, for the rest I've put a quick source 
tarball here:

http://ipython.scipy.org/dist/testing/ipython-0.6.13_cvs.tar.gz

Thanks,

f