[IPython-user] paste frustrations

Robin robince@gmail....
Wed Mar 4 10:04:24 CST 2009


On Wed, Mar 4, 2009 at 3:23 PM, Robin <robince@gmail.com> wrote:
> On Wed, Mar 4, 2009 at 3:10 PM, Robin <robince@gmail.com> wrote:
>> On Wed, Mar 4, 2009 at 2:47 PM, Ludwig Schwardt
>> <ludwig.schwardt@gmail.com> wrote:
>>> Interesting...
>>> This error persists in many configurations. I've pasted Robin's example text
>>> with the following results:
>>> Mac OS 10.5.6
>>> ----------------------
>>> System Python 2.5.1: 70% MemoryError, 30% syntax error (no success)
>>> ipython 0.9.1 based on System Python 2.5.1: 90% MemoryError (+crash), 10%
>>> syntax error (no success)
>>> Mac OS 10.4.11
>>> ------------------------
>>> System Python 2.3.5: 100% success
>>> Universal Python 2.4.4 from http://www.pythonmac.org/packages/py24-fat: 70%
>>> success, 30% syntax error
>>> Universal Python 2.5 from http://www.pythonmac.org/packages/py25-fat: 80%
>>> success, 20% syntax error
>>> ipython 0.8.2 based on Universal Python 2.5: 70% success, 30% syntax error
>>> I also used a more structured sequence of characters, to probe the
>>> boundaries on which this problem occurs. I created a 2048-character string
>>> without newlines, which worked fine. When I split the string into 32 lines
>>> of 64-character strings by inserting newlines, the problems started.
>>> I tried all three line endings (LF, CR+LF and CR) by saving the text in vi
>>> with the desired format, using pbcopy to load the file into the clipboard
>>> and verifying the line endings by "pbpaste | hexdump".
>>> They all caused the same error.
>>> Analysis
>>> ------------
>>> - The error only occurs on Mac OS X (both Tiger and Leopard), and with
>>> Python version at least 2.4.
>>> - On Tiger it sometimes works, while on Leopard it always breaks.
>>> - The error only occurs when pasting multi-line text of sufficient length.
>>> - The error manifests as repeating sections of text (of variable length).
>>> These sections are repeats of the text starting from just after the last
>>> newline.
>>> - Sometimes the point at which corrupted text is inserted in the pasted text
>>> is 1024 characters from the start of the pasted text (which might indicate
>>> an internal buffer boundary).
>>> - The error affects Python rather than ipython. It results in either a
>>> SyntaxError or a MemoryError (parser stack overflow). The latter triggers a
>>> crash in ipython.
>>> - The error seems to be independent of line ending.
>>> - The error appears non-deterministic or with unobvious dependencies. When
>>> Python is started from scratch, the text can sometimes be successfully
>>> pasted more than 20 times without triggering an error. Other times the error
>>> appears immediately.
>>> - The error is independent of the source of the pasted text.
>>> - The error is independent of the terminal application.
>>> I suspect that some buffer in the Python line parser is not cleared
>>> properly.
>>> Ludwig
>>
>> Wow, thanks for the comprehensive analysis!
>>
>> Motivated by the way you considered the probabilities - I revisited other apps.
>>
>> Pasting into vim or another editor never fails... but when I tried cat
>> << EOF again I think it also fails there (I guess I was (un)lucky the
>> first time I tried that) about 90% of the time.
>>
>> Could you see if cat << EOF works for you - I guess this suggests it
>> might be something with readline again? (and not a python issue)
>>
>
> Although strangely
> echo "
> <paste>
>
> works reliably...
>
> I don't know enough about low level console stuff to know what the
> difference between cat << EOF and echo " is? << use readline?
>

Right, that was wrong, both cat << and echo " give corruption

In fact even cat > test which I'm pretty sure doesn't use readline at
all still has some problems (although much fewer and they seem more
deterministic).

Sorry that this is getting off topic for ipython now... but it is
probably the same problem as :
https://svn.enthought.com/epd/ticket/324
so I guess it will continue to come up from time to time.

Cheers

Robin


More information about the IPython-user mailing list