[IPython-dev] coercing to Unicode error in IPython.MultiEngineClient

Brian Granger ellisonbg.net@gmail....
Thu Jun 12 15:05:20 CDT 2008


Doug,

We haven't thought about this yet and it is a big issue.  I will make
another post about this that we can all use to work out this issue.

Brian

On Thu, Jun 12, 2008 at 1:33 PM, Doug Jones <dfj225@gmail.com> wrote:
> Hi Brian,
>
> I ran into some more issues installing IPython on a different machine. This
> machine runs Python 2.4 and it seems that some of the IPython code is now
> using the 'with' statement, which I don't think is included in Python 2.4.
> Is 2.4 no longer supported?
>
> The trace from my setup.py install attempt has been copied at the end of the
> message.
>
> Thanks,
> ~Doug
>
> Note: I'd already executed a python setup.py build without error.
>
>  python setup.py install --prefix=$HOME/local
> ============================================================================
> BUILDING IPYTHON
>                 python: 2.4.4 (#1, Jul 29 2007, 19:42:10)  [GCC 4.1.1
>                         (Gentoo 4.1.1-r3)]
>               platform: linux2
>
> OPTIONAL DEPENDENCIES
>         Zope.Interface: yes
>                Twisted: 2.5.0
>               Foolscap: 0.2.8
>                OpenSSL: 0.6
>                 sphinx: 0.3
>               pygments: 0.10
>                   nose: 0.10.3
>                pexpect: 0.999
> running install
> running build
> running build_py
> running build_scripts
> running install_lib
> byte-compiling
> /home/djones/local/lib/python2.4/site-packages/IPython/config/config.py to
> config.pyc
>   File
> "/home/djones/local/lib/python2.4/site-packages/IPython/config/config.py",
> line 49
>     with raw(self):
>            ^
> SyntaxError: invalid syntax
> byte-compiling
> /home/djones/local/lib/python2.4/site-packages/IPython/kernel/contexts.py to
> contexts.pyc
>   File
> "/home/djones/local/lib/python2.4/site-packages/IPython/kernel/contexts.py",
> line 171
>     with parallel as pr:
>                 ^
> SyntaxError: invalid syntax
> running install_scripts
> changing mode of /home/djones/local/bin/pycolor to 755
> changing mode of /home/djones/local/bin/ipengine to 755
> changing mode of /home/djones/local/bin/ipcontroller to 755
> changing mode of /home/djones/local/bin/ipcluster to 755
> changing mode of /home/djones/local/bin/ipython to 755
> changing mode of /home/djones/local/bin/irunner to 755
> running install_data
> Traceback (most recent call last):
>   File "setup.py", line 174, in ?
>     setup(**setup_args)
>   File "/usr/lib/python2.4/distutils/core.py", line 149, in setup
>     dist.run_commands()
>   File "/usr/lib/python2.4/distutils/dist.py", line 946, in run_commands
>     self.run_command(cmd)
>   File "/usr/lib/python2.4/distutils/dist.py", line 966, in run_command
>     cmd_obj.run()
>   File "/usr/lib/python2.4/distutils/command/install.py", line 510, in run
>     self.run_command(cmd_name)
>   File "/usr/lib/python2.4/distutils/cmd.py", line 333, in run_command
>     self.distribution.run_command(command)
>   File "/usr/lib/python2.4/distutils/dist.py", line 966, in run_command
>     cmd_obj.run()
>   File "/home/djones/ipython_src/ipython/setupext/install_data_ext.py", line
> 75, in run
>     (out, _) = self.copy_file(f, dir)
>   File "/usr/lib/python2.4/distutils/cmd.py", line 376, in copy_file
>     dry_run=self.dry_run)
>   File "/usr/lib/python2.4/distutils/file_util.py", line 117, in copy_file
>     if not os.path.isfile(src):
>   File "/usr/lib/python2.4/posixpath.py", line 208, in isfile
>     st = os.stat(path)
> TypeError: coercing to Unicode: need string or buffer, list found
>
>
>
>
> On Wed, Jun 11, 2008 at 4:49 PM, Brian Granger <ellisonbg.net@gmail.com>
> wrote:
>>
>> > Connecting with the MultiEngineClient is working now. It seems that it
>> > was
>> > simply from passing the wrong arguments to the constructor that I was
>> > having
>> > problems before.
>> >
>> > All in all, I think the furl files are an improvement and the addition
>> > of
>> > secure connections is certainly appreciated.
>>
>> Thanks for the feedback.  We _really_ needed to address the security
>> issue and I think Foolscap and the furls really work well.  They give
>> a high level of security, but make it pretty easy to setup.
>>
>> Brian
>>
>> > Thanks,
>> > ~doug
>> >
>> > On Tue, Jun 10, 2008 at 4:22 PM, Brian Granger <ellisonbg.net@gmail.com>
>> > wrote:
>> >>
>> >> > Thanks for the information. I thought I had used the configuration
>> >> > creation
>> >> > command to create .furl files in my home directory. Right now, i'm
>> >> > using
>> >> > mpiexec to start the engines, so perhaps this is creating some sort
>> >> > of
>> >> > issue
>> >> > with the shell environment the engines are created in. I'll double
>> >> > check
>> >> > and
>> >> > let you know if I continue to have problems.
>> >>
>> >> Hmm, if you are editing the ipython config files to set the locations
>> >> of the furl files, that should work.  Maybe the only thing you need to
>> >> change is how you are creating the MultiEngineClient.
>> >>
>> >> It no longer takes the (ip, port) tuple.  It now takes the name of the
>> >> furl file, or if empty it will use the location given in the config
>> >> files.
>> >>
>> >> Keep me posted.
>> >>
>> >> Brian
>> >>
>> >> > ~doug
>> >> >
>> >> >
>> >> > On Tue, Jun 10, 2008 at 3:55 PM, Brian Granger
>> >> > <ellisonbg.net@gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> This is not a unicode error, but rather, our API for starting the
>> >> >> client has changed due to the new security stuff.  This is the stuff
>> >> >> that I am working on documenting as we speak.
>> >> >>
>> >> >> Here is a minimal description:
>> >> >>
>> >> >> 1) When the controller now starts, it creates a set of files (in
>> >> >> your
>> >> >> .ipython directory by default)
>> >> >>
>> >> >> ipcontroller-tc.furl
>> >> >> ipcontroller-mec.furl
>> >> >> ipcontroller-engine.furl
>> >> >>
>> >> >> 2)  These files contain a secure URL that 1) tells the engine and
>> >> >> clients where the controller is running and 2) gives the engine and
>> >> >> clients authority to connect to the controller in a secure manner.
>> >> >>
>> >> >> 3) To use these files, they have to be available to the client and
>> >> >> engines when they start.  The easiest way of handling this is to
>> >> >>
>> >> >> a) copy ipcontroller-engine.furl to the .ipython directory on the
>> >> >> machine(s) where the engines will run
>> >> >>
>> >> >> b) copy ipcontroller-tc and -mec to the .ipython dir of the machine
>> >> >> where the clients will run.
>> >> >>
>> >> >> Then everything will "just work".  By this, I mean that you can
>> >> >> create
>> >> >> the clients with no arguments:
>> >> >>
>> >> >> mec = client.MultiEngineClient()
>> >> >>
>> >> >> If you have put the .furl files in different locations you can do:
>> >> >>
>> >> >> mec =
>> >> >>
>> >> >> client.MultiEngineClient('/Users/me/furlfiles/ipcontroller-mec.furl')
>> >> >>
>> >> >> In all of this, you can think of the furl files as being keys (just
>> >> >> like a house key) that grants an entity access to a particular
>> >> >> resource.  The controller creates the keys and the engines/client
>> >> >> must
>> >> >> present/use them to use the capabilities of the controller.
>> >> >>
>> >> >> See if you can use this to get things working.  The big benefit of
>> >> >> using all this is:
>> >> >>
>> >> >> 1) Users don't have to track what ip/port the controller is running
>> >> >> on.
>> >> >>
>> >> >> 2) Everything is secure by default - authentication + encryption (if
>> >> >> you have pyOpenSSL installed
>> >> >>
>> >> >> 3) The controller now uses random port numbers, making it even more
>> >> >> difficult for hostiles to discover.
>> >> >>
>> >> >> Let me know if you have any more problems getting this working.  We
>> >> >> would love feedback if you have ideas of how things could be made
>> >> >> easier.
>> >> >>
>> >> >> Also, check out the new command line flags on the ipcontroller and
>> >> >> ipengine scripts to control where the furl files are created and
>> >> >> looked for.
>> >> >>
>> >> >> Cheers,
>> >> >>
>> >> >> Brian
>> >> >>
>> >> >> On Tue, Jun 10, 2008 at 12:57 PM, Doug Jones <dfj225@gmail.com>
>> >> >> wrote:
>> >> >> > Hi all,
>> >> >> >
>> >> >> > I tried a simple test of the latest IPython branch and the
>> >> >> > MultiEngineClient. When I attempted to connect to a running
>> >> >> > cluster
>> >> >> > on
>> >> >> > my
>> >> >> > local machine, I got the following error:
>> >> >> >
>> >> >> >  mec = client.MultiEngineClient(('localhost', 10105))
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > ---------------------------------------------------------------------------
>> >> >> > TypeError                                 Traceback (most recent
>> >> >> > call
>> >> >> > last)
>> >> >> >
>> >> >> > /home/djones/svn/basin_remote/trunk/scripts/<ipython console> in
>> >> >> > <module>()
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > /home/djones/local/lib64/python2.5/site-packages/IPython/kernel/client.pyc
>> >> >> > in get_multiengine_client(furl_or_file)
>> >> >> >      67     """
>> >> >> >      68     client =
>> >> >> > blockingCallFromThread(_client_tub.get_multiengine_client,
>> >> >> > ---> 69         furl_or_file)
>> >> >> >      70     return client.adapt_to_blocking_client()
>> >> >> >      71
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > /home/djones/local/lib64/python2.5/site-packages/IPython/kernel/twistedutil.pyc
>> >> >> > in blockingCallFromThread(f, *a, **kw)
>> >> >> >      97                 result.raiseException()
>> >> >> >      98             except Exception, e:
>> >> >> > ---> 99                 raise e
>> >> >> >     100         return result
>> >> >> >     101
>> >> >> >
>> >> >> > TypeError: coercing to Unicode: need string or buffer, tuple found
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > Thanks,
>> >> >> > ~Doug
>> >> >> >
>> >> >> > _______________________________________________
>> >> >> > IPython-dev mailing list
>> >> >> > IPython-dev@scipy.org
>> >> >> > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>> >> >> >
>> >> >> >
>> >> >
>> >> >
>> >
>> >
>
>


More information about the IPython-dev mailing list