Defining custom types

Travis Oliphant oliphant at ee.byu.edu
Thu Oct 26 19:15:08 CDT 2006


> It's just confusing as the documentation indicates that the setitem 
> function should return 0 for success and a negative number for 
> failure. But within Array_FromPyScalar, we have:
>
>         ret->descr->f->setitem(op, ret->data, ret);
>
>         if (PyErr_Occurred()) {
>                 Py_DECREF(ret);
>                 return NULL;
>         } else {
>                 return (PyObject *)ret;
>         }
>
I see the problem.  We are assuming an error is set on failure, so both 
-1 should be returned and an error condition set for your own setitem 
function.    This is typical Python behavior.   I'll fix the documentation.

>
>     >
>     > I seem to be able to load values into the array, but I can't extract
>     > anything out of the array, even to print it. In gdb I've
>     verified that
>     > loading DateTime.now() correctly puts a float representation of the
>     > date into my array. However, if I try to get the value out, I get an
>     > error:
>     > >>> mxArr[0] = DateTime.now()
>     > >>> mxArr[0]
>     > Traceback (most recent call last):
>     >   File "<stdin>", line 1, in ?
>     >   File "/usr/lib/python2.4/site-packages/numpy/core/numeric.py",
>     line
>     > 391, in array_repr
>     >     ', ', "array(")
>     >   File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py",
>     > line 204, in array2string
>     >     separator, prefix)
>     >   File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py",
>     > line 160, in _array2string
>     >     format = _floatFormat(data, precision, suppress_small)
>     >   File "/usr/lib/python2.4/site-packages/numpy/core/arrayprint.py",
>     > line 281, in _floatFormat
>     >     non_zero = _uf.absolute(data.compress(_uf.not_equal(data, 0)))
>     > TypeError: bad operand type for abs()
>     >
>     > I'm not sure why it's trying to call abs() on my object to print it.
>
>     Because that's the implication of inheriting from a double.  It's
>     just
>     part of the code that tries to format your values into an array
>     (notice
>     the _floatFormat).  I actually borrowed this code from numarray so I
>     can't speak to exactly what it's doing without more study.
>
>
> Hmm, so does Numpy ignore the tp_repr and tp_str fields in the 
> PyTypeObject of the underlying type?  admittedly haven't had a chance 
> to look at this code closely yet.


How arrays print is actually user-settable.  The default printing 
function does indeed ignore tp_repr and tp_str of the underlying scalar 
objects in order to be able to set precisions. 

Now, we could probably fix the default printing function to actually use 
the tp_repr and/or tp_str fields of the corresponding scalar objects. 

This is worth filing a ticket about.

In the mean time you can create a new array print function that checks 
for your data-type as the type of the array and then does something 
different otherwise it calls the old function.   Then, register this 
function as the print function for arrays.

>
>
>     > I have a separate PyNumberMethods attached to my object type, copied
>     > from the float scalar type, and nb_absolute is set to 0. When I
>     break
>     > at the various functions I've registered, the last thing Numpy
>     tries
>     > to do is cast my custom data type to an object type (which it
>     does so
>     > successfully) via _broadcast_cast.
>
>     Don't confuse the Python object  associated when an element of the
>     array
>     is extracted and the data-type of the array. Also don't confuse the
>     PyNumberMethods of the scalar object with the ufuncs.  Defining
>     PyNumberMethods won't usually give you the ability to calculate
>     ufuncs.
>
>
> Okay, is my understanding here correct? I am defining two type 
> descriptors:
> PyArray_Descr mxNumpyType - describes the Numpy array type.
> PyTypeObject mxNumpyDataType - describes the data type of the contents 
> of the array (i.e. mxNumpyType->typeobj points to this), inherits from 
> PyDoubleArrType_Type and overrides some fields as mentioned above.
>
The nomenclature is that mxNumPyType is the data-type of the array and 
your PyTypeObject is the "type" of the elements of the array.   So, you 
have the names a bit backward. 

So, to correspond with the way I use the words "type" and "data-type", I 
would name them:

PyArray_Descr mxNumpyDataType
PyTypeObject mxNumpyType

> And the getitem and setitem functions are designed to only give/take 
> PyObject* of type mxDateTime.
>
These are in the 'f' member of the PyArray_Descr structure, so 
presumably you have also filled in your PyArray_Descr structure with 
items from PyArray_DOUBLE?

> I guess it's not clear to me whether the abs() referred to by the 
> error is an abs() ufunc or the nb_absolute pointer in the 
> PyNumberMethods. Let me try overriding ufuncs and get back to you...
>
>     Perhaps you just want to construct an "object" array of mxDateTime's.
>     What is the reason you want to define an mxDateTime data-type?
>
>
> Currently I am using an object array of mxDateTime's, but it's rather 
> frustrating that I can't treat them as normal floats internally since 
> that's really all they are.


Ah, I see.   So, you would like to be able to say view the array of 
mxDateTimes as an array of "floats" (using the .view method).  You are 
correct that this doesn't make sense when you are talking about objects, 
but might if mxDateTime objects are really just floats. 

I just wanted to make sure you were aware of the object array route.  
The new data-type route is less well traveled but I'm anxious to smooth 
the wrinkles out.   Your experiences will help.  

Bascially we are moving from Numeric being a "builtin data-types" only 
to a NumPy that has "arbitrary" data-types with a few special-cased 
"builtins"   We need more experience to clarify the issues.  Your 
identification of problems in the default printing, for example, is one 
thing that will help.

Keep us posted.  I'd love to here how things went and what can be done 
to improve.

-Travis


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642




More information about the Numpy-discussion mailing list