[SciPy-dev] 'O' type arrays and containers as values...

Fernando Perez Fernando.Perez at colorado.edu
Fri Nov 4 15:26:50 CST 2005

Travis Oliphant wrote:

>>In [5]: a=scipy.empty((2,2),'O')
>>In [6]: a[0,0] = (1,2,3)
> The problem here is the ambiguity of the left hand side and how 
> assignment is generally done.   There is no special-check for this 
> case.   All that happens is that (1,2,3) gets converted to an object 
> array and the elements copied over.  So, you are trying to do the 
> equivalent of
> a[0,0] = array((1,2,3),'O')
> But, the right-hand side is converted to a length-3 object array --- 
> with container types it's ambiguous as to what you really want for the 
> object array. 

Buit I'd argue that if the LHS is of type 'O', even that should work, no? 
Basically my way of looking at this is: type 'O' arrays hold opaque pointers 
to arbitrary Python objects.  In a sense they are almost like a dictionary 
with tuple keys, but with the usual semantics of scipy arrays.

Since the index given is a single-element index, then the object on the RHS 
should be assigned to that 'pointer' in the table, no?

> Then of course the length-3 array cannot be copied into the result.  
> Now, one solution is to special-case the PyArrayObject assignment and 
> check for single-index assigment and just copy whatever the value is 
> directly over.   Of course the more special-cases, the slower all code 
> gets.

I realize that this is unpleasant, but as they stand now, the 'O' arrays are 
rather strange (as I just found out).  They can hold many python objects as 
their values, even some containers (lists are OK), but not tuples, lists or 
other arrays.  I think that kind of special-casing should be avoided.  Since 
single-element assignment for the 'O' type _is_ special (the arrays can 
contain inhomogeneous entries already):

In [11]: a = scipy.array(['hi Travis',3.14,scipy.cos],'O')

In [12]: a
Out[12]: array([hi Travis, 3.14, <ufunc 'cos'>], dtype=object)

then I think that this should work.

Because it gets worse:

In [20]: a = scipy.array([[0,(1,2,3)],[2,3]],'O')

In [21]: a.shape
Out[21]: (2, 2)

In [22]: a[0,1]
Out[22]: (1, 2, 3)

In [23]: a[0,1] = (1,2,3)
exceptions.ValueError                                Traceback (most recent 
call last)


ValueError: number of elements in destination must be integer multiple of 
number of elements in source

This shows that an 'O' array can in fact _contain_ tuples if given at 
construction time, but you can't assign to them later.  IMHO this qualifies as 
a wart in capital letters :)

Can you somehow just special-case the = operator for 'O' at the 'top level', 
so that it doesn't do anything at all to the RHS?  As I see it, the RHS should 
just be held unmodified, in essence taking the address of the underlying 
PyObject pointer and little more...  But I don't really know that code in 
detail, so I may be missing something here.

I realize that the fix may be a bit painful, and I'm willing to look at ways 
of finding the least-nasty solution, but I honestly think this needs fixing. 
It's too bizarre of a special behavior to live in the Python world...

> This has always been a problem.  You would get a similar error in Numeric.

I actually saw it in Numeric, but preferred to report for scipy core.  I want 
you to keep your word that 24.1 is the last of Numeric :)



More information about the Scipy-dev mailing list