[IPython-user] Problem with using the descriptor protocol in ipython

Fernando Perez Fernando.Perez at colorado.edu
Thu Dec 29 19:08:14 CST 2005


Hi Matt,

I'm cc-ing the list, because a) I want this well archived, including google b) 
it is a delicate issue that may also bite other users in the future.

Matt's message follows in full, for context.  My reply is below.

Feinstein, Matthew R. wrote:
> Hi Fernando--
> 
> I was playing around with the descriptor protocol & found something
> peculiar going on in ipython (0.6.15)
> (on Windows, invoked as C:\Python24\Scripts\python24\python ipython
> -pylab)--
> 
> Wrote the following program (in the file 'descriptors.py'):
> 
> class report_access(object):
> 	def __init__(self, val):
> 		self.val = val
> 	
> 	def __get__(self, obj, objtype):
> 		print 'Getting', self, obj, objtype
> 		print obj.y
> 		obj.y  = obj.y + 1
> 		return self.val
> 		
> 	def __set__(self, obj, val):
> 		print 'Setting', type(obj)
> 		self.val = val
> 		
> class watched(object):
> 	x = report_access([1,2,3])
> 	y = 5
> 
> And then, in ipython:
> 
> In [3]: import descriptors
> 
> In [4]: m = descriptors.watched()
> 
> In [5]: m.x
> Getting <descriptors.report_access object at 0x01E23310>
> <descriptors.watched ob
> ject at 0x01DDDCB0> <class 'descriptors.watched'>
> 5
> Getting <descriptors.report_access object at 0x01E23310>
> <descriptors.watched ob
> ject at 0x01DDDCB0> <class 'descriptors.watched'>
> 6
> Out[5]: [1, 2, 3]
> 
> In [6]: m.y
> Out[6]: 7
> 
> So it looks like the __get__ method is accessed -twice-. (!). This
> doesn't happen with the __set__ method, or with the standard python
> interpreter.
> 
> Matt Feinstein

The basic issue is the following: when you type in ipython x.y.z, it tries to 
analyze that input to understand whether this refers to an alias, a magic 
command, a python variable, or is undefined.  But to study your input without 
actually calling eval() on it (which was the old approach, with similar 
problems to yours but far, far worse), it breaks up the input into components, 
and carefully walks all the possible namespaces looking for things of the form 
'x', then 'y', then 'z'.  But inevitably, this means accessing (via getattr()) 
'y' on object 'x', and the result is the problem you report.

I've been able to fix the issue, for the case when autocall is OFF.  As of 
now, ipython will guarantee that, _if autocall is OFF_, objects will never be 
introspected with getattr, eval() or any other mechanism which can potentially 
cause side effects, before the line is actually executed.

The situation with attributes and side effects does remain delicate, though. 
You need to be aware of two cases:

1. In ipython, if autocall is ON (the historical default, easy to toggle at 
runtime with %autocall or permanently in your ipythonrc file), side-effects 
are still possible.

2. In any interactive python shell with readline support for tab-completion 
(_including the default CPYthon '>>>' interpreter), side-effects can be 
triggered.  This is because the completion call makes getattr() calls on the 
objects it is trying to complete for.

The following illustrates your problem in the plain python shell, with 
readline loaded and configured for tab-completion.

I typed 'm.x.' and then hit TAB.  The following happened:

 >>>
 >>> m.x.Getting <descriptors.report_access object at 0x4007282c> 
<descriptors.watched object at 0x40072a0c> <class 'descriptors.watched'>
6

   File "<stdin>", line 1
     m.x.
        ^
SyntaxError: invalid syntax
 >>>


So at least now, I've (I think finally) found a way to ensure that getattr() 
will never be invoked when autocall is off, while retaining the rest of 
ipython's functionality (aliases, magics,etc.)  But you need to remain aware 
of the two issues outlined above.

Thanks for bringing this to my attention.

Cheers,

f




More information about the IPython-user mailing list