[IPython-dev] PyMAD/IPython which list updated

Frédéric Mantegazza mantegazza at ill.fr
Wed Mar 23 06:57:10 CST 2005

Le Dimanche 20 Mars 2005 12:20, Fernando Perez a écrit :

> Thanks for this update.  Below follow a few more comments, it would be
> great if you could keep the doc updated as we hash out the issues.  I'm
> thinking a wiki would be great for this, but for now we don't have one. 
> So while I code, you can be the FrédéWiki ;-)  I actually think most
> things will be cleared up in short order, so in reality there's no need
> to do much.

I've tried and implemented some of the new features. And I updated the 
FrédéWiki ;o)

> 1.
> Following this, IPython now has a facility for custom exception handling.
> IPython can take a list of custom exceptions and a handler, and call that
> handler if any exception in the list matches.  Note that this can only be
> done for a single handler, because I can't (short of doing dynamic code
> generation, which I don't want to get into)  have multiple except
> clauses.  So your one handler would get the exception info for _any_ of
> the matched exceptions, and it would have to deal with them.

Ok, works fine :o)

> 2.
> Done.  A new rc parameter, readline_merge_completions, controls this
> (true by default).
> New completers can now be added by using the
> __IPYTHON__.set_custom_completer method.  Plase see its docstring for
> details.

Working too :o)

> 3.
> OK, this is easy.  Just make sure that the objects IPython sees have a
> .getdoc() method; I've added a call to that into my object introspection
> code. Have a look at inspect.getdoc, you can probably have your .getdoc
> method be like that, since it does a good job of cleaning up indentation
> for the docstring.  Your method should conform to the same API as
> inspect.getdoc: return None if nothing is found, return a string
> (formatted to your liking) otherwise.
> In fact, the change is already in CVS, so feel free to play with it.  See
> OInspect.py's getdoc() function for details.

It seems to work, but I need some more explanation of the usage, and how 
things work.

I'm using the inspect.getdoc() code in my Spectro object. But I would like 
to have something more accurate. To sumarize:

obj? should give me the docstring of the class definition of obj.
obj.method? should give me the docstring of obj.method().

Is it possible ?

> 4.
> Alternatively, with the custom exception handler mechanism (see above),
> you can just catch specifically the exceptions you'd like to treat
> differently and deal with them directly.  Which of the two approaches you
> take will depend exactly on what you want to do.

Yes, this is enough for our needs for the moment.

> 5.
Last, I tried to use pre-filters, to avoid user deleting some of our 
objects, but I didn't find documentation about them. Where is it ?

> 6.
Prompt is also working fine, but I get some strange behaviour when using 
multi-lines prompt. Yes, I think that you don't support multi-line prompt, 
but it works, except:

- Going up in the history change the color of the prompt (turn back to 
- The indentation for outputs should be computed with the lenght of the last 
  line of the prompt, and not with all the prompt.

This is just a remark, but it may not be difficult to solve these problems.

> 7.
> Why can't you just use the push() method?  That's pretty much what it
> does, see its docstring.  If this is not enough, you'll need to clarify
> to me exactly why not, and what else you'd need.

See my question on the user ML. I have problems using it.

> 8.
In fact, my first idea before making a dynamic prompt was to have a status 
bar (dynamically refreshed) at the bottom of IPython, as it is done in 
minicom, for example. Is it difficult to implement ?


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://projects.scipy.org/pipermail/ipython-dev/attachments/20050323/4e91b4e9/attachment.html 
-------------- next part --------------
This document describes the interactions needed by PyMAD with IPython.

    Frédéric Mantegazza, 

:Version: $Rev: 1324 $

1) Introduction

The philosophy of **PyMAD** is to give to the user a high-level set of tools to
drive a spectrometer. The idea is to directly make some internal python
objects available to the user, and let him combine them.

So, The final user just calls some methods of a few high-level objects to drive
the spectrometer. This as the advantage to make all the python scripting stuff
available, to build some new high level tools.

As we also need a system which can be used from several places, we use a
client/server framework, with **Pyro**. **Pyro** is a distributed objects
server. It just make some remote python objects available to a client as if they
where local.

2) User interaction

To avoid the need for the final user to write python scripts and run them to do
something, we need a simple working environment which gives the possibility to
interactively use the server objects. That's where **IPython** solves lots of
problems !

**IPython** is an enhanced python shell. It let the user runs python code, but
has many powerfull features :

- history, even between sessions
- colorized and customizable traceback
- code completion
- magic commands
- much more...
**IPython** is on the client-side of **PyMAD**. In fact, there is a special
client which connects to the remote objects of **PyMAD**, launch an embbeded
**IPython** instance, and make the remote objects available to the user, in the
global namespace of **IPython**. This way, the user can drive the spectrometer
through the methods of these objects, or can build complex scripts to do complex

**PyMAD** also use the magic commands to make a simple command interpreter.
The magic functions use TPG (Toy Parser Generator), a easy-to-use parser based
on regexp. These MAD-like commands are for users which don't know about python,
but also to make shortcuts, to avoid the need to write several lines of normal
python code to do some complex but repetitive tasks.

One important point is that **PyMAD** can understand both syntax, which can be
combined. Most of the time, simple commands will be used, but python code can
be more powerfull to do expert measures (with automatic feedback interaction
according to the results), or to prototype a new complex command.

3) **IPython** interactions

In all the following, ipshell is a IPShellEmbed instance.

Here is the main **PyMAD/IPython** interactions:

1. Catch custom *PyMADError* exceptions. The solution (from CVS version), is to
   use the new set_custom_sxc() method of __IP object (ie ishell.IP object)::

    def pymadHandler(self, exceptionType, exceptionValue, traceback):
        """ This is a special handler to handle custom exceptions in IPython.
        # Handle PyMAD exceptions
        if issubclass(exceptionType, PyMADError):
        # Handle Pyro exceptions
        elif issubclass(exceptionType, Pyro.errors.ConnectionClosedError):
            MessageLogger().critical("Connexion with server lost. Please restart it...")
            if self.code_to_run_src.find('.') != -1:
                code = self.code_to_run_src.split(".")
                obj = eval(code[0], self.user_ns)
                MessageLogger().warning("Server found again. Running last command...")
                eval(self.code_to_run_src, self.user_ns)
                MessageLogger().warning("Could not automatically rebind to server. Better restart console")
        # Handle AttributeError exception
        elif exceptionType is AttributeError:
            if self.code_to_run_src.find('.') != -1:
                code = self.code_to_run_src.split(".")
                obj = eval(code[0], self.user_ns)
                if isinstance(obj, Pyro.core.DynamicProxyWithAttrs):
                    MessageLogger().error("%s has no attribute %s" % (code[0], code[1]))
        # Others
            print "\n\n"
            print "*** Unknown exception ***"
            print "Exception type :", exceptionType
            print "Exception value:", exceptionValue
            print "Traceback      :", traceback
            print "Source code    :", self.code_to_run_src

    ipshell.IP.set_custom_exc((PyMADError, Pyro.errors.ConnectionClosedError, AttributeError), pymadHandler)

2. Add some new matchers for completion. As **PyMAD** uses remote objects,
   completion only shows the client Pyro proxy. So we create a new matcher which
   get the object (from the text param), call a special method on this object
   which returns all available attributes (in fact, only these we want to show
   to the user). Here is the code used::

    def proxy_matches(self, text, state):
        """ Get the attributes of a remove Pyro object.
        message = MessageLogger()
        # Another option, seems to work great. Catches things like ''.<tab>
        m = re.match(r"(\S+(\.\w+)*)\.(\w*)$", text)
        matches = []
        if m:
            expr, attr = m.group(1, 3)
                obj = eval(expr, self.namespace)
                if isinstance(obj, Pyro.core.DynamicProxyWithAttrs):
                    words = obj.getAvailableAttributes()
                    n = len(attr)
                    if words:
                        for word in words:
                            if word[:n] == attr and word != "__builtins__":
                                matches.append("%s.%s" % (expr, word))
            except NameError:
            except Pyro.errors.ConnectionClosedError:
                message.critical("Connexion with server lost.")
        return matches

    ipshell.IP.Completer.merge_completions = False

3. In the same way as matchers, get the docstring from the remote object instead
   of the client one when using 'obj?' and 'obj.method?' syntax. This can be
   done by adding a getdoc() method to the object. But it only works for the
   first syntax. And if object provides __call__ method, it adds some more infos
   not wanted.

4. New exception handler. Here, the idea is to be able to present different kind
   of exceptions in different ways. Some will only print a simple message, some
   others will print the entire traceback (maybe a modified traceback).
   This is done in the point 1.

5. Prevent objects from being deleted by *del* keyword. This can be done with
   pre-filters, but I didn't find how such filters are defined.

6. Dynamic Prompt. Works fine using the PEP 215 syntax. When using multi-line
   prompt, color changed when going up in the history. Also, the indentation for
   output should be computed from the lenght of the last prompt line, and not
   the whole prompt. Here is the prompt used::

    argv = ["-pi1",
            "Ki=${globals()['Spectro'].Ki} Kf=${globals()['Spectro'].Kf}\n\
            mode=${globals()['Spectro'].driveMode} flipper=${globals()['Spectro'].flipperBeam}\n\
            PyMAD@${os.path.expandvars('$PYRO_NS_HOSTNAME')}>>> ",
            " ",
7. Access to the command-line interpreter, to have **IPython** interprets code
   has if it was entered through keyboard (ie make difference between magic
   commands and normal python code). Can be done with the ipshell.IP.push()
   method, but it does not work. This will be used in the magic 'do' method.
   At the moment, it can only interprets magic commands (using magic() method).

8. Status bar at the bottom line. Like prompt, should be dynamically refreshde.

Open points are 3 (for obj.method? syntax), 5, 6 (for multi-line prompt), 7, 8.

More information about the IPython-dev mailing list