[IPython-user] Several problems with cp720 (Arabic Windows-1256)
Jörgen Stenarson
jorgen.stenarson@bostream...
Fri Jan 4 11:53:28 CST 2008
Diaa Sami skrev:
> Hi,
> I'm having problems with IPython with all versions after 0.7.3, and I just found that there are
> problems with the most recent pyreadline too.
> I'm on an English Windows XP system with Arabic language support(which means that
> sys.stdin.encoding='cp720'), Control Panel > Regional and Language options > Advanced tab > Arabic
> (Egypt) selected.
> You can find the problematic cases below
> 1- I upgraded to a recent IPython(0.8.1) but I left the old pyreadline-1.3, IPython starts properly
> but whenever I enter any text I get the following error:
>
Diaa,
The problem is that current versions of python does not support cp720.
After some googling it seems there are fixes in the trunk for python but
that doesn't help us.
For me (using swedish windows) the default codepage of the cmd.exe is
cp850. But there is a mismatch to the encoding used in the filesystem
which makes it more convenient for me to always launch cmd.exe such that
it switches to cp1252 using the command chcp 1252. Perhaps you could try
to do the same but with chcp 1256 to get arabic.
You could also try adding:
import pyreadline.unicode_helper
pyreadline.unicode_helper.pyreadline_codepage="cp1256"
to force pyreadline to use another codepage than the default, however
this may not work if there is a mismatch between 1256 and 720.
Please let us know how it goes
/Jörgen
> ---- start here
> In [1]: f
> ---------------------------------------------------------------------------
> LookupError Traceback (most recent call last)
>
> C:\Python25\Lib\site-packages\IPython\iplib.py in raw_input(self, prompt, contin
> ue_prompt)
> 2049
> 2050 try:
> -> 2051 line = raw_input_original(prompt).decode(self.stdin_encoding
> )
> 2052 except ValueError:
> 2053 warn("\n********\nYou or a %run:ed script called sys.stdin.c
> lose()"
>
> LookupError: unknown encoding: cp720
> ---- end here
>
> I solved this problem by changing line 380 in iplib.py to
> self.stdin_encoding = 'ascii'
> instead of
> self.stdin_encoding = sys.stdin.encoding or 'ascii'
>
> 2- I upgraded to the current IPython(0.8.2) and using the latest pyreadline-1.5, when starting
> IPython I get the following error:
>
> ---- start here
> ---------------------------------------------------------------------------
> LookupError Python 2.5.1: C:\Python25\python.exe
> Mon Dec 24 16:33:25 2007
> A problem occured executing Python code. Here is the sequence of function
> calls leading up to the error, with the most recent (innermost) call last.
>
> C:\Python25\scripts\ipython in <module>()
> 12 IPython.Shell.IPShell().mainloop(sys_exit=1)
> 13
> 14 [or simply IPython.Shell.IPShell().mainloop(1) ]
> 15
> 16 and IPython will be your working environment when you start python. The
> final
> 17 sys.exit() call will make python exit transparently when IPython finishe
> s, so
> 18 you don't have an extra prompt to get out of.
> 19
> 20 This is probably useful to developers who manage multiple Python version
> s and
> 21 don't want to have correspondingly multiple IPython versions. Note that
> in
> 22 this mode, there is no way to pass IPython any command-line options, as
> those
> 23 are trapped first by Python itself.
> 24 """
> 25
> 26 import IPython
> ---> 27 IPython.Shell.start().mainloop()
> global IPython.Shell.start.mainloop = undefined
> 28
> 29
> 30
> 31
> 32
> 33
> 34
> 35
> 36
> 37
> 38
> 39
> 40
> 41
> 42
>
> C:\Python25\lib\site-packages\IPython\Shell.py in start(user_ns=None)
> 1137 arg1 = argv[1]
> 1138 if arg1.endswith('-gthread'):
> 1139 shell = IPShellGTK
> 1140 elif arg1.endswith( '-qthread' ):
> 1141 shell = IPShellQt
> 1142 elif arg1.endswith( '-q4thread' ):
> 1143 shell = IPShellQt4
> 1144 elif arg1.endswith('-wthread'):
> 1145 shell = IPShellWX
> 1146 elif arg1.endswith('-pylab'):
> 1147 shell = _matplotlib_shell_class()
> 1148 else:
> 1149 shell = IPShell
> 1150 else:
> 1151 shell = IPShell
> -> 1152 return shell(user_ns = user_ns)
> shell = <class IPython.Shell.IPShell at 0x00FB6990>
> user_ns = None
> 1153
> 1154 # Some aliases for backwards compatibility
> 1155 IPythonShell = IPShell
> 1156 IPythonShellEmbed = IPShellEmbed
> 1157 #************************ End of file <Shell.py> ***********************
> ****
> 1158
> 1159
> 1160
> 1161
> 1162
> 1163
> 1164
> 1165
> 1166
> 1167
>
> C:\Python25\lib\site-packages\IPython\Shell.py in __init__(self=<IPython.Shell.I
> PShell instance at 0x00AF3F80>, argv=None, user_ns=None, user_global_ns=None, de
> bug=1, shell_class=<class 'IPython.iplib.InteractiveShell'>)
> 59 # Tag when runcode() is active, for exception handling
> 60 CODE_RUN = None
> 61
> 62 #-----------------------------------------------------------------------
> ------
> 63 # This class is trivial now, but I want to have it in to publish a clean
>
> 64 # interface. Later when the internals are reorganized, code that uses th
> is
> 65 # shouldn't have to change.
> 66
> 67 class IPShell:
> 68 """Create an IPython instance."""
> 69
> 70 def __init__(self,argv=None,user_ns=None,user_global_ns=None,
> 71 debug=1,shell_class=InteractiveShell):
> 72 self.IP = make_IPython(argv,user_ns=user_ns,
> 73 user_global_ns=user_global_ns,
> ---> 74 debug=debug,shell_class=shell_class)
> debug = 1
> shell_class = <class 'IPython.iplib.InteractiveShell'>
> 75
> 76 def mainloop(self,sys_exit=0,banner=None):
> 77 self.IP.mainloop(banner)
> 78 if sys_exit:
> 79 sys.exit()
> 80
> 81 #-----------------------------------------------------------------------
> ------
> 82 def kill_embedded(self,parameter_s=''):
> 83 """%kill_embedded : deactivate for good the current embedded IPython
> .
> 84
> 85 This function (after asking for confirmation) sets an internal flag
> so that
> 86 an embedded IPython will never activate again. This is useful to
> 87 permanently disable a shell that is being called inside a loop: once
> you've
> 88 figured out what you needed from it, you may then kill it and the pr
> ogram
> 89 will then continue to run without the interactive shell interfering
> again.
>
> C:\Python25\Lib\site-packages\IPython\ipmaker.py in make_IPython(argv=[r'C:\Pyth
> on25\scripts\ipython'], user_ns=None, user_global_ns=None, debug=1, rc_override=
> None, shell_class=<class 'IPython.iplib.InteractiveShell'>, embedded=False, **kw
> ={})
> 745 IP_rc.banner = 0
> 746 if IP_rc.banner:
> 747 BANN_P = IP.BANNER_PARTS
> 748 else:
> 749 BANN_P = []
> 750
> 751 if IP_rc.profile: BANN_P.append('IPython profile: %s\n' % IP_rc.prof
> ile)
> 752
> 753 # add message log (possibly empty)
> 754 if msg.summary: BANN_P.append(msg.summary)
> 755 # Final banner is a string
> 756 IP.BANNER = '\n'.join(BANN_P)
> 757
> 758 # Finalize the IPython instance. This assumes the rc structure is f
> ully
> 759 # in place.
> --> 760 IP.post_config_initialization()
> IP.post_config_initialization = <bound method InteractiveShell.post_conf
> ig_initialization of <IPython.iplib.InteractiveShell object at 0x00AF5750>>
> 761
> 762 return IP
> 763 #************************ end of file <ipmaker.py> *********************
> *****
> 764
> 765
> 766
> 767
> 768
> 769
> 770
> 771
> 772
> 773
> 774
> 775
>
> C:\Python25\Lib\site-packages\IPython\iplib.py in post_config_initialization(sel
> f=<IPython.iplib.InteractiveShell object at 0x00AF5750>)
> 650 This is called after the configuration files have been processed
> to
> 651 'finalize' the initialization."""
> 652
> 653 rc = self.rc
> 654
> 655 # Object inspector
> 656 self.inspector = OInspect.Inspector(OInspect.InspectColors,
> 657 PyColorize.ANSICodeColors,
> 658 'NoColor',
> 659 rc.object_info_string_level)
>
> 660
> 661 self.rl_next_input = None
> 662 self.rl_do_indent = False
> 663 # Load readline proper
> 664 if rc.readline:
> --> 665 self.init_readline()
> self.init_readline = <bound method InteractiveShell.init_readline of <IP
> ython.iplib.InteractiveShell object at 0x00AF5750>>
> 666
> 667
> 668 # local shortcut, this is used a LOT
> 669 self.log = self.logger.log
> 670
> 671 # Initialize cache, set in/out prompts and printing system
> 672 self.outputcache = CachedOutput(self,
> 673 rc.cache_size,
> 674 rc.pprint,
> 675 input_sep = rc.separate_in,
> 676 output_sep = rc.separate_out,
> 677 output_sep2 = rc.separate_out2,
> 678 ps1 = rc.prompt_in1,
> 679 ps2 = rc.prompt_in2,
> 680 ps_out = rc.prompt_out,
>
> C:\Python25\Lib\site-packages\IPython\iplib.py in init_readline(self=<IPython.ip
> lib.InteractiveShell object at 0x00AF5750>)
> 1348 # is being used (as on Leopard) the readline config is
> 1349 # not run as the syntax for libedit is different.
> 1350 if not readline.uses_libedit:
> 1351 for rlcommand in self.rc.readline_parse_and_bind:
> 1352 readline.parse_and_bind(rlcommand)
> 1353
> 1354 # remove some chars from the delimiters list
> 1355 delims = readline.get_completer_delims()
> 1356 delims = delims.translate(string._idmap,
> 1357 self.rc.readline_remove_delims)
> 1358 readline.set_completer_delims(delims)
> 1359 # otherwise we end up with a monster history after a while:
> 1360 readline.set_history_length(1000)
> 1361 try:
> 1362 #print '*** Reading readline history' # dbg
> -> 1363 readline.read_history_file(self.histfile)
> readline.read_history_file = <bound method Readline.read_history_file of
> <pyreadline.rlmain.Readline object at 0x00E73A90>>
> self.histfile = 'C:\\Documents and Settings\\Administrator\\_ipython\\hi
> story'
> 1364 except IOError:
> 1365 pass # It doesn't exist yet.
> 1366
> 1367 atexit.register(self.atexit_operations)
> 1368 del atexit
> 1369
> 1370 # Configure auto-indent for all platforms
> 1371 self.set_autoindent(self.rc.autoindent)
> 1372
> 1373 def ask_yes_no(self,prompt,default=True):
> 1374 if self.rc.quiet:
> 1375 return True
> 1376 return ask_yes_no(prompt,default)
> 1377
> 1378 def _should_recompile(self,e):
>
> C:\Python25\Lib\site-packages\pyreadline\rlmain.py in read_history_file(self=<py
> readline.rlmain.Readline object at 0x00E73A90>, filename=r'C:\Documents and Sett
> ings\Administrator\_ipython\history')
> 168
> 169 def set_history_length(self, length):
> 170 '''Set the number of lines to save in the history file.
> 171
> 172 write_history_file() uses this value to truncate the history fil
> e
> 173 when saving. Negative values imply unlimited history file size.
> 174 '''
> 175 self._history.set_history_length(length)
> 176
> 177 def clear_history(self):
> 178 '''Clear readline history'''
> 179 self._history.clear_history()
> 180
> 181 def read_history_file(self, filename=None):
> 182 '''Load a readline history file. The default filename is ~/.hist
> ory.'''
> --> 183 self._history.read_history_file(filename)
> self._history.read_history_file = <bound method LineHistory.read_history
> _file of <pyreadline.lineeditor.history.LineHistory object at 0x00E73BD0>>
> filename = 'C:\\Documents and Settings\\Administrator\\_ipython\\history
> '
> 184
> 185 def write_history_file(self, filename=None):
> 186 '''Save a readline history file. The default filename is ~/.hist
> ory.'''
> 187 self._history.write_history_file(filename)
> 188
> 189 #Completer functions
> 190
> 191 def set_completer(self, function=None):
> 192 '''Set or remove the completer function.
> 193
> 194 If function is specified, it will be used as the new completer
> 195 function; if omitted or None, any completer function already
> 196 installed is removed. The completer function is called as
> 197 function(text, state), for state in 0, 1, 2, ..., until it retur
> ns a
> 198 non-string value. It should return the next possible completion
>
> C:\Python25\Lib\site-packages\pyreadline\lineeditor\history.py in read_history_f
> ile(self=<pyreadline.lineeditor.history.LineHistory object at 0x00E73BD0>, filen
> ame=r'C:\Documents and Settings\Administrator\_ipython\history')
> 55
> 56 history_length=property(get_history_length,set_history_length)
> 57 history_cursor=property(get_history_cursor,set_history_cursor)
> 58
> 59 def clear_history(self):
> 60 '''Clear readline history.'''
> 61 self.history[:] = []
> 62 self.history_cursor = 0
> 63
> 64 def read_history_file(self, filename=None):
> 65 '''Load a readline history file.'''
> 66 if filename is None:
> 67 filename=self.history_filename
> 68 try:
> 69 for line in open(filename, 'r'):
> ---> 70 self.add_history(lineobj.ReadLineTextBuffer(ensure_unico
> de(line.rstrip())))
> self.add_history = <bound method LineHistory.add_history of <pyreadline.
> lineeditor.history.LineHistory object at 0x00E73BD0>>
> global lineobj.ReadLineTextBuffer = <class 'pyreadline.lineeditor.lineob
> j.ReadLineTextBuffer'>
> global ensure_unicode = <function ensure_unicode at 0x00C91630>
> line.rstrip = <built-in method rstrip of str object at 0x00FE6908>
> 71 except IOError:
> 72 self.history = []
> 73 self.history_cursor = 0
> 74
> 75 def write_history_file(self, filename=None):
> 76 '''Save a readline history file.'''
> 77 if filename is None:
> 78 filename=self.history_filename
> 79 fp = open(filename, 'wb')
> 80 for line in self.history[-self.history_length:]:
> 81 fp.write(ensure_str(line.get_line_text()))
> 82 fp.write('\n')
> 83 fp.close()
> 84
> 85
>
> C:\Python25\Lib\site-packages\pyreadline\unicode_helper.py in ensure_unicode(tex
> t='i1, i2=6,26')
> 5 # Distributed under the terms of the BSD License. The full license is
> in
> 6 # the file COPYING, distributed as part of this software.
> 7 #***********************************************************************
> ******
> 8 import sys
> 9
> 10 try:
> 11 pyreadline_codepage=sys.stdout.encoding
> 12 except AttributeError: #This error occurs when pdb imports readli
> ne and doctest has replaced
> 13 #stdout with stdout collector
> 14 pyreadline_codepage="ascii" #assume ascii codepage
> 15
> 16
> 17 def ensure_unicode(text):
> 18 """helper to ensure that text passed to WriteConsoleW is unicode"""
> 19 if isinstance(text, str):
> ---> 20 return text.decode(pyreadline_codepage, "replace")
> text.decode = <built-in method decode of str object at 0x00FE6DB8>
> global pyreadline_codepage = 'cp720'
> 21 return text
> 22
> 23 def ensure_str(text):
> 24 """Convert unicode to str using pyreadline_codepage"""
> 25 if isinstance(text, unicode):
> 26 return text.encode(pyreadline_codepage, "replace")
> 27 return text
> 28
> 29
> 30
> 31
> 32
> 33
> 34
> 35
>
> LookupError: unknown encoding: cp720
>
> **********************************************************************
>
> Oops, IPython crashed. We do our best to make it stable, but...
>
> A crash report was automatically generated with the following information:
> - A verbatim copy of the crash traceback.
> - A copy of your input history during this session.
> - Data on your current IPython configuration.
>
> It was left in the file named:
> 'C:\Documents and Settings\Administrator\_ipython\IPython_crash_report.t
> xt'
> If you can email this file to the developers, the information in it will help
> them in understanding and correcting the problem.
>
> You can mail it to: Ville Vainio at vivainio@gmail.com
> with the subject 'IPython Crash Report'.
>
> If you want to do it now, the following command will work (under Unix):
> mail -s 'IPython Crash Report' vivainio@gmail.com < C:\Documents and Settings\Ad
> ministrator\_ipython\IPython_crash_report.txt
>
> To ensure accurate tracking of this issue, please file a report about it at:
> http://projects.scipy.org/ipython/ipython/report
> ---- end here
>
> I solved this error by installing pyreadline-1.4.4 instead.
>
> So apparently the first error is a problem with IPython itself but the latter is a problem with
> pyreadline.
>
> I'd appreciate if these problem can be solved for future versions of IPython(without having to
> modify it manually), I'm ready to do some testing if needed.
>
> Thanks
>
> Regards,
> Diaa M. Sami
>
> _______________________________________________
> IPython-user mailing list
> IPython-user@scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-user
>
More information about the IPython-user
mailing list