[IPython-dev] !commands in the zmq model
Sat Aug 28 12:22:26 CDT 2010
This is really great news. The simple implementation that was there
covered the basic cases, but left a lot to be desired.
On Sat, Aug 28, 2010 at 12:29 AM, Fernando Perez <firstname.lastname@example.org> wrote:
> I've spent hours on getting a better implementation of
> in the multiprocess model, that:
> 1. shows output as it happens
> 2. is cleanly interruptible.
This is really wonderful. Thanks for putting in the effort on this.
> We can interrupt subprocesses fine with our current
> (subprocess.Popen-based) implementation, but we can't get their output
> cleanly as it happens. So in the usual:
> for i in range(200):
> print i,
> print 'done!'
> We only see *all* the numbers at the very end. I haven't found any way
> around this with subprocess, and neither did our old ipythonx
> implementation done by Gael; as best as I understand popen it simply
> can't be done: in pipe mode, the C stdio library does block-buffered
> io on all process execution and there's no clean way I can find, to
> read from a pipe with a timeout.
Bummer, I thought it was smarter than that, but oh well.
> But the good news is that I think I have an implementation that will
> work on *nix (linux/mac), using pexpect. On Windows we'll have to
> fall back to subprocess.popen(), with its limitations.
I think this is a very reasonable compromise. Maybe it will motivate
some passionate Windows users to look further at a Windows pexpect ;-)
> In addition, the pexpect implementation gives us the benefit of
> correctly formatted 'ls' output, reverting the surprise Brian had
> initially from ls being only on one column. Pexpect creates a proper
> pseudo-tty, and knows how to read from it very intelligently.
This is awesome!
> It's a shame it doesn't exist on windows, but it's apparently a very
> non-trivial task to port it (I remember hearing William Stein several
> times comment on how good it would be for Sage to have pexpect on
> Windows, and knowing them, if it was doable they would have already
> done it).
> I may not finish this today, I'm too exhausted, but if anyone knows
> this type of problem well, pitch in. I'm sure I'll make good use of
> any help...
I am going to be focusing on the GUI stuff and ZMQ/PyZMQ stuff this
weekend. I will be around on IRC and working if you want to chat.
> ps - for reference, the current implementation is along the lines of:
> from __future__ import print_function
> import sys
> import pexpect
> self = get_ipython()
> if 1:
> def system( cmd):
> cmd = self.var_expand(cmd, depth=2).strip()
> sh = '/bin/bash'
> timeout = 0.05 # seconds
> pcmd = '%s -c %r' % (sh, cmd)
> child = pexpect.run(pcmd, logfile=sys.stdout)
> ## child = pexpect.spawn(sh, ['-c', cmd])
> ## while True:
> ## res = child.expect([pexpect.TIMEOUT, pexpect.EOF], timeout)
> ## if res==0:
> ## #pass
> ## print(child.before, end='')
> ## elif res==1:
> ## break
> except KeyboardInterrupt:
> print('\nInterrupted command: %r.' % cmd, file=sys.stderr)
> #return child
> IPython-dev mailing list
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
More information about the IPython-dev