[IPython-user] Several problems with cp720 (Arabic Windows-1256)

Diaa Sami diaasami@gmail....
Wed Dec 26 19:10:15 CST 2007


Hi,
I'm having problems with IPython with all versions after 0.7.3, and I just found that there are
problems with the most recent pyreadline too.
I'm on an English Windows XP system with Arabic language support(which means that
sys.stdin.encoding='cp720'), Control Panel > Regional and Language options > Advanced tab > Arabic
(Egypt) selected.
You can find the problematic cases below
1- I upgraded to a recent IPython(0.8.1) but I left the old pyreadline-1.3, IPython starts properly
but whenever I enter any text I get the following error:

---- start here
In [1]: f
---------------------------------------------------------------------------
LookupError                               Traceback (most recent call last)

C:\Python25\Lib\site-packages\IPython\iplib.py in raw_input(self, prompt, contin
ue_prompt)
    2049
    2050         try:
-> 2051             line = raw_input_original(prompt).decode(self.stdin_encoding
)
    2052         except ValueError:
    2053             warn("\n********\nYou or a %run:ed script called sys.stdin.c
lose()"

LookupError: unknown encoding: cp720
---- end here

I solved this problem by changing line 380 in iplib.py to
self.stdin_encoding = 'ascii'
instead of
self.stdin_encoding = sys.stdin.encoding or 'ascii'

2- I upgraded to the current IPython(0.8.2) and using the latest pyreadline-1.5, when starting
IPython I get the following error:

---- start here
---------------------------------------------------------------------------
LookupError                            Python 2.5.1: C:\Python25\python.exe
                                                    Mon Dec 24 16:33:25 2007
A problem occured executing Python code.  Here is the sequence of function
calls leading up to the error, with the most recent (innermost) call last.

C:\Python25\scripts\ipython in <module>()
      12 IPython.Shell.IPShell().mainloop(sys_exit=1)
      13
      14 [or simply IPython.Shell.IPShell().mainloop(1) ]
      15
      16 and IPython will be your working environment when you start python. The
final
      17 sys.exit() call will make python exit transparently when IPython finishe
s, so
      18 you don't have an extra prompt to get out of.
      19
      20 This is probably useful to developers who manage multiple Python version
s and
      21 don't want to have correspondingly multiple IPython versions. Note that
in
      22 this mode, there is no way to pass IPython any command-line options, as
those
      23 are trapped first by Python itself.
      24 """
      25
      26 import IPython
---> 27 IPython.Shell.start().mainloop()
         global IPython.Shell.start.mainloop = undefined
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      39
      40
      41
      42

C:\Python25\lib\site-packages\IPython\Shell.py in start(user_ns=None)
    1137         arg1 = argv[1]
    1138         if arg1.endswith('-gthread'):
    1139             shell = IPShellGTK
    1140         elif arg1.endswith( '-qthread' ):
    1141             shell = IPShellQt
    1142         elif arg1.endswith( '-q4thread' ):
    1143             shell = IPShellQt4
    1144         elif arg1.endswith('-wthread'):
    1145             shell = IPShellWX
    1146         elif arg1.endswith('-pylab'):
    1147             shell = _matplotlib_shell_class()
    1148         else:
    1149             shell = IPShell
    1150     else:
    1151         shell = IPShell
-> 1152     return shell(user_ns = user_ns)
         shell = <class IPython.Shell.IPShell at 0x00FB6990>
         user_ns = None
    1153
    1154 # Some aliases for backwards compatibility
    1155 IPythonShell = IPShell
    1156 IPythonShellEmbed = IPShellEmbed
    1157 #************************ End of file <Shell.py> ***********************
****
    1158
    1159
    1160
    1161
    1162
    1163
    1164
    1165
    1166
    1167

C:\Python25\lib\site-packages\IPython\Shell.py in __init__(self=<IPython.Shell.I
PShell instance at 0x00AF3F80>, argv=None, user_ns=None, user_global_ns=None, de
bug=1, shell_class=<class 'IPython.iplib.InteractiveShell'>)
      59 # Tag when runcode() is active, for exception handling
      60 CODE_RUN = None
      61
      62 #-----------------------------------------------------------------------
------
      63 # This class is trivial now, but I want to have it in to publish a clean

      64 # interface. Later when the internals are reorganized, code that uses th
is
      65 # shouldn't have to change.
      66
      67 class IPShell:
      68     """Create an IPython instance."""
      69
      70     def __init__(self,argv=None,user_ns=None,user_global_ns=None,
      71                  debug=1,shell_class=InteractiveShell):
      72         self.IP = make_IPython(argv,user_ns=user_ns,
      73                                user_global_ns=user_global_ns,
---> 74                                debug=debug,shell_class=shell_class)
         debug = 1
         shell_class = <class 'IPython.iplib.InteractiveShell'>
      75
      76     def mainloop(self,sys_exit=0,banner=None):
      77         self.IP.mainloop(banner)
      78         if sys_exit:
      79             sys.exit()
      80
      81 #-----------------------------------------------------------------------
------
      82 def kill_embedded(self,parameter_s=''):
      83     """%kill_embedded : deactivate for good the current embedded IPython
.
      84
      85     This function (after asking for confirmation) sets an internal flag
so that
      86     an embedded IPython will never activate again.  This is useful to
      87     permanently disable a shell that is being called inside a loop: once
  you've
      88     figured out what you needed from it, you may then kill it and the pr
ogram
      89     will then continue to run without the interactive shell interfering
again.

C:\Python25\Lib\site-packages\IPython\ipmaker.py in make_IPython(argv=[r'C:\Pyth
on25\scripts\ipython'], user_ns=None, user_global_ns=None, debug=1, rc_override=
None, shell_class=<class 'IPython.iplib.InteractiveShell'>, embedded=False, **kw
={})
     745         IP_rc.banner = 0
     746     if IP_rc.banner:
     747         BANN_P = IP.BANNER_PARTS
     748     else:
     749         BANN_P = []
     750
     751     if IP_rc.profile: BANN_P.append('IPython profile: %s\n' % IP_rc.prof
ile)
     752
     753     # add message log (possibly empty)
     754     if msg.summary: BANN_P.append(msg.summary)
     755     # Final banner is a string
     756     IP.BANNER = '\n'.join(BANN_P)
     757
     758     # Finalize the IPython instance.  This assumes the rc structure is f
ully
     759     # in place.
--> 760     IP.post_config_initialization()
         IP.post_config_initialization = <bound method InteractiveShell.post_conf
ig_initialization of <IPython.iplib.InteractiveShell object at 0x00AF5750>>
     761
     762     return IP
     763 #************************ end of file <ipmaker.py> *********************
*****
     764
     765
     766
     767
     768
     769
     770
     771
     772
     773
     774
     775

C:\Python25\Lib\site-packages\IPython\iplib.py in post_config_initialization(sel
f=<IPython.iplib.InteractiveShell object at 0x00AF5750>)
     650         This is called after the configuration files have been processed
  to
     651         'finalize' the initialization."""
     652
     653         rc = self.rc
     654
     655         # Object inspector
     656         self.inspector = OInspect.Inspector(OInspect.InspectColors,
     657                                             PyColorize.ANSICodeColors,
     658                                             'NoColor',
     659                                             rc.object_info_string_level)

     660
     661         self.rl_next_input = None
     662         self.rl_do_indent = False
     663         # Load readline proper
     664         if rc.readline:
--> 665             self.init_readline()
         self.init_readline = <bound method InteractiveShell.init_readline of <IP
ython.iplib.InteractiveShell object at 0x00AF5750>>
     666
     667
     668         # local shortcut, this is used a LOT
     669         self.log = self.logger.log
     670
     671         # Initialize cache, set in/out prompts and printing system
     672         self.outputcache = CachedOutput(self,
     673                                         rc.cache_size,
     674                                         rc.pprint,
     675                                         input_sep = rc.separate_in,
     676                                         output_sep = rc.separate_out,
     677                                         output_sep2 = rc.separate_out2,
     678                                         ps1 = rc.prompt_in1,
     679                                         ps2 = rc.prompt_in2,
     680                                         ps_out = rc.prompt_out,

C:\Python25\Lib\site-packages\IPython\iplib.py in init_readline(self=<IPython.ip
lib.InteractiveShell object at 0x00AF5750>)
    1348             # is being used (as on Leopard) the readline config is
    1349             # not run as the syntax for libedit is different.
    1350             if not readline.uses_libedit:
    1351                 for rlcommand in self.rc.readline_parse_and_bind:
    1352                     readline.parse_and_bind(rlcommand)
    1353
    1354             # remove some chars from the delimiters list
    1355             delims = readline.get_completer_delims()
    1356             delims = delims.translate(string._idmap,
    1357                                       self.rc.readline_remove_delims)
    1358             readline.set_completer_delims(delims)
    1359             # otherwise we end up with a monster history after a while:
    1360             readline.set_history_length(1000)
    1361             try:
    1362                 #print '*** Reading readline history'  # dbg
-> 1363                 readline.read_history_file(self.histfile)
         readline.read_history_file = <bound method Readline.read_history_file of
  <pyreadline.rlmain.Readline object at 0x00E73A90>>
         self.histfile = 'C:\\Documents and Settings\\Administrator\\_ipython\\hi
story'
    1364             except IOError:
    1365                 pass  # It doesn't exist yet.
    1366
    1367             atexit.register(self.atexit_operations)
    1368             del atexit
    1369
    1370         # Configure auto-indent for all platforms
    1371         self.set_autoindent(self.rc.autoindent)
    1372
    1373     def ask_yes_no(self,prompt,default=True):
    1374         if self.rc.quiet:
    1375             return True
    1376         return ask_yes_no(prompt,default)
    1377
    1378     def _should_recompile(self,e):

C:\Python25\Lib\site-packages\pyreadline\rlmain.py in read_history_file(self=<py
readline.rlmain.Readline object at 0x00E73A90>, filename=r'C:\Documents and Sett
ings\Administrator\_ipython\history')
     168
     169     def set_history_length(self, length):
     170         '''Set the number of lines to save in the history file.
     171
     172         write_history_file() uses this value to truncate the history fil
e
     173         when saving. Negative values imply unlimited history file size.
     174         '''
     175         self._history.set_history_length(length)
     176
     177     def clear_history(self):
     178         '''Clear readline history'''
     179         self._history.clear_history()
     180
     181     def read_history_file(self, filename=None):
     182         '''Load a readline history file. The default filename is ~/.hist
ory.'''
--> 183         self._history.read_history_file(filename)
         self._history.read_history_file = <bound method LineHistory.read_history
_file of <pyreadline.lineeditor.history.LineHistory object at 0x00E73BD0>>
         filename = 'C:\\Documents and Settings\\Administrator\\_ipython\\history
'
     184
     185     def write_history_file(self, filename=None):
     186         '''Save a readline history file. The default filename is ~/.hist
ory.'''
     187         self._history.write_history_file(filename)
     188
     189     #Completer functions
     190
     191     def set_completer(self, function=None):
     192         '''Set or remove the completer function.
     193
     194         If function is specified, it will be used as the new completer
     195         function; if omitted or None, any completer function already
     196         installed is removed. The completer function is called as
     197         function(text, state), for state in 0, 1, 2, ..., until it retur
ns a
     198         non-string value. It should return the next possible completion

C:\Python25\Lib\site-packages\pyreadline\lineeditor\history.py in read_history_f
ile(self=<pyreadline.lineeditor.history.LineHistory object at 0x00E73BD0>, filen
ame=r'C:\Documents and Settings\Administrator\_ipython\history')
      55
      56     history_length=property(get_history_length,set_history_length)
      57     history_cursor=property(get_history_cursor,set_history_cursor)
      58
      59     def clear_history(self):
      60         '''Clear readline history.'''
      61         self.history[:] = []
      62         self.history_cursor = 0
      63
      64     def read_history_file(self, filename=None):
      65         '''Load a readline history file.'''
      66         if filename is None:
      67             filename=self.history_filename
      68         try:
      69             for line in open(filename, 'r'):
---> 70                 self.add_history(lineobj.ReadLineTextBuffer(ensure_unico
de(line.rstrip())))
         self.add_history = <bound method LineHistory.add_history of <pyreadline.
lineeditor.history.LineHistory object at 0x00E73BD0>>
         global lineobj.ReadLineTextBuffer = <class 'pyreadline.lineeditor.lineob
j.ReadLineTextBuffer'>
         global ensure_unicode = <function ensure_unicode at 0x00C91630>
         line.rstrip = <built-in method rstrip of str object at 0x00FE6908>
      71         except IOError:
      72             self.history = []
      73             self.history_cursor = 0
      74
      75     def write_history_file(self, filename=None):
      76         '''Save a readline history file.'''
      77         if filename is None:
      78             filename=self.history_filename
      79         fp = open(filename, 'wb')
      80         for line in self.history[-self.history_length:]:
      81             fp.write(ensure_str(line.get_line_text()))
      82             fp.write('\n')
      83         fp.close()
      84
      85

C:\Python25\Lib\site-packages\pyreadline\unicode_helper.py in ensure_unicode(tex
t='i1, i2=6,26')
       5 #  Distributed under the terms of the BSD License.  The full license is
in
       6 #  the file COPYING, distributed as part of this software.
       7 #***********************************************************************
******
       8 import sys
       9
      10 try:
      11     pyreadline_codepage=sys.stdout.encoding
      12 except AttributeError:        #This error occurs when pdb imports readli
ne and doctest has replaced
      13                               #stdout with stdout collector
      14     pyreadline_codepage="ascii"   #assume ascii codepage
      15
      16
      17 def ensure_unicode(text):
      18     """helper to ensure that text passed to WriteConsoleW is unicode"""
      19     if isinstance(text, str):
---> 20         return text.decode(pyreadline_codepage, "replace")
         text.decode = <built-in method decode of str object at 0x00FE6DB8>
         global pyreadline_codepage = 'cp720'
      21     return text
      22
      23 def ensure_str(text):
      24     """Convert unicode to str using pyreadline_codepage"""
      25     if isinstance(text, unicode):
      26         return text.encode(pyreadline_codepage, "replace")
      27     return text
      28
      29
      30
      31
      32
      33
      34
      35

LookupError: unknown encoding: cp720

**********************************************************************

Oops, IPython crashed. We do our best to make it stable, but...

A crash report was automatically generated with the following information:
   - A verbatim copy of the crash traceback.
   - A copy of your input history during this session.
   - Data on your current IPython configuration.

It was left in the file named:
         'C:\Documents and Settings\Administrator\_ipython\IPython_crash_report.t
xt'
If you can email this file to the developers, the information in it will help
them in understanding and correcting the problem.

You can mail it to: Ville Vainio at vivainio@gmail.com
with the subject 'IPython Crash Report'.

If you want to do it now, the following command will work (under Unix):
mail -s 'IPython Crash Report' vivainio@gmail.com < C:\Documents and Settings\Ad
ministrator\_ipython\IPython_crash_report.txt

To ensure accurate tracking of this issue, please file a report about it at:
http://projects.scipy.org/ipython/ipython/report
---- end here

I solved this error by installing pyreadline-1.4.4 instead.

So apparently the first error is a problem with IPython itself but the latter is a problem with
pyreadline.

I'd appreciate if these problem can be solved for future versions of IPython(without having to
modify it manually), I'm ready to do some testing if needed.

Thanks

Regards,
Diaa M. Sami



More information about the IPython-user mailing list