[IPython-User] Advice on using a prefilter

Aaron Meurer asmeurer@gmail....
Tue Aug 7 00:06:20 CDT 2012

On Mon, Aug 6, 2012 at 10:47 PM, Aaron Meurer <asmeurer@gmail.com> wrote:
> On Mon, Aug 6, 2012 at 6:14 PM, Thomas Kluyver <takowl@gmail.com> wrote:
>> Hi Aaron,
>> I've realised after writing the comments below that it can mostly be
>> summarised as: sorry, we haven't been maintaining that API very well,
>> and you might just be hitting the limits of it. It's been on my to-do
>> list for a while, I'll try to find time to look at it. The detailed
>> version follows.
>> On 7 August 2012 00:39, Aaron Meurer <asmeurer@gmail.com> wrote:
>>> - What is the purpose of the continue_prompt argument to transform()?
>> It relates to multi-line inputs - possibly it's True if the input
>> spans multiple lines. Or maybe it should get called with one line at a
>> time, and continue_prompt is True for lines that aren't the first in
>> the input. Sorry I can't be clearer, this part of the code is quite
>> complex, and I think the way we use it has changed.
> If what you say is correct, then this simply doesn't work.
>>> - How do I make this work for multi-line inputs.  So far, I have something like
>>> In [1]: 1/2
>>> Out[1]: 1/2
>>> In [2]: def f():
>>>    ...:     return 1/2
>>>    ...:
>>> In [3]: f()
>>> Out[3]: 0.5
>>> The transformer doesn't seem to be called at all if the input is more
>>> than one line (it doesn't seem to matter if this is because it's a
>>> block or just because of a line continuation).
>> Yes, that would be due to this bit of code:
>> https://github.com/ipython/ipython/blob/master/IPython/core/interactiveshell.py#L2593
> So can this if statement be removed?

This works, sort of.  I really need to be able to process the whole
cell at once.  Otherwise, it's not possible for me to properly
tokenize something like

In [1]: (1/2 +
   ...: 2)

So clearly the line by line api should be replaced with a cell by cell api.

Aaron Meurer

>> There are two modules handling transforming the input: inputsplitter
>> does only static transformations like %magic calls (with the %
>> escape), and prefilter does dynamic transformations like aliases and
>> automagics. Prefilter is only called when the input is a single line.
>> I've been meaning for some time to refactor these two modules into a
>> more coherent arrangement, but it's complex code right at the heart of
>> IPython, and it mostly works at the moment, so I've found other things
>> to do. It might be that what you're trying to do isn't fully possible
>> yet; maybe this will give me the impetus to actually work on it.
>>> - If the input is bad, it raises a SyntaxError, but on the transformed
>>> expresion.  This is expected, but I'd like to avoid it for user
>>> experience purposes. What is the best way to fall back to the original
>>> input in the case of a SyntaxError?
>> I think the current model is that your code should only transform suitable input
> Fair enough.  Do you happen to know what's the best way to tell if a
> given line of code is syntactically correct (obviously without
> executing it)?
>>> - The code at https://bitbucket.org/birkenfeld/ipython-physics/src/71b2d850da00/physics.py
>>> claims that the transformer object is supposed to subclass
>>> PrefilterTransformer, but it doesn't work.  I also didn't get it to
>>> work.  If I do, I get
>> It looks like the instantiation expects that it will be passed the
>> shell and prefilter_manager objects, but they're optional parameters
>> anyway, probably because they can be set by the config system. If you
>> instantiate within the load_ipython_extension call, you can probably
>> pass ip and ip.prefilter_manager to it.
> Yes, I'm using ip.prefilter_manager.register_transformer, the same as
> the physics file.
> So should I open an issue or issues?  I'm not really clear on what is
> and isn't a bug, and how things are supposed to work, or how they
> should work in an improved API.
> Aaron Meurer

More information about the IPython-User mailing list