[IPython-user] Fwd: Is ipython too nosy?

Brian Granger ellisonbg.net@gmail....
Thu Mar 1 16:12:44 CST 2007

> > BTW just as a half baked idea -- have you considered just running
> > tokenizer.generate_tokens over the input lines (provided they don't start with
> > "!") for ipython and work from that? I haven't found some quick overview of
> > the syntax extensions ipython supports, but I would suspect the ones I am
> > aware of could be implemented on top of a tokenstream from generate_tokens[1]
> > and that this might make for simpler and more robust code and possibly a nicer
> > interface for syntactic extensions. One benefit is that you'd get future
> > lexical syntax supported for free, the other would be that IIRC in the past
> > there have been possibly confusing bugs that resulted from ipython tokenizing
> > things in a somewhat ad hoc manner at unexepcted occassions (such as this
> > one), which presumably mostly involved getting confused by strings.

I had a number of good discussions with people at Pycon last week
about moving in this direction with ipython's extended syntax.  The
main challenge is that ipython's syntax is not a static grammar.  This
is partly what makes it so powerful and yet simple.  But it does
complicate parsing it.


More information about the IPython-user mailing list