Defining custom types

Travis Oliphant oliphant.travis at ieee.org
Fri Oct 27 15:02:56 CDT 2006


Jonathan Wang wrote:
> On 10/27/06, *Travis Oliphant* <oliphant.travis at ieee.org 
> <mailto:oliphant.travis at ieee.org>> wrote:
>
>     > If I redefine the string function, I encounter another, perhaps more
>     > serious problem leading to a segfault. I've defined my string
>     function
>     > to be extremely simple:
>     > >>> def printer(arr):
>     > ...   return str(arr[0])
>     >
>     > Now, if I try to print an element of the array:
>     > >>> mxArr[0]
>     >
>     > I get to this stack trace:
>     > #0  scalar_value (scalar=0x814be10, descr=0x5079e0) at
>     > scalartypes.inc.src:68
>     > #1  0x0079936a in PyArray_Scalar (data=0x814cf98, descr=0x5079e0,
>     > base=0x814e7a8) at arrayobject.c:1419
>     > #2  0x007d259f in array_subscript_nice (self=0x814e7a8,
>     op=0x804eb8c)
>     > at arrayobject.c:1985
>     > #3  0x00d17dde in PyObject_GetItem (o=0x814e7a8, key=0x804eb8c) at
>     > Objects/abstract.c:94
>     >
>     > (Note: for some reason gdb claims that arrayobject.c:1985 is
>     > array_subscript_nice, but looking at my source this line is
>     actually
>     > in array_item_nice. *boggle*)
>     >
>     > But scalar_value returns NULL for all non-native types. So,
>     destptr in
>     > PyArray_Scalar is set to NULL, and the call the copyswap segfaults.
>     >
>     > Perhaps scalar_value should be checking the scalarkind field of
>     > PyArray_Descr, or using the elsize and alignment fields to
>     figure out
>     > the pointer to return if scalarkind isn't set?
>
>     Hmmm... It looks like the modifications to scalar_value did not take
>     into account user-defined types.  I've added a correction so that
>     user-defined types will use setitem to set the scalar value into the
>     array.  Presumably your setitem function can handle setting the array
>     with scalars of your new type?
>
>     I've checked the changes into SVN. 
>
>
> Do there also need to be changes in scalartypes.inc.src to use getitem 
> if a user-defined type does not inherit from a Numpy scalar?
This needs to be clarified.  I don't think it's possible to do it 
without inheriting from a numpy scalar at this point (the void numpy 
scalar can be inherited from and is pretty generic).  I know I was not 
considering that case when I wrote the code.
> i.e. at scalartypes.inc.src:114 we should return some pointer 
> calculated from the PyArray_Descr's elsize and alignment field to get 
> the destination for the "custom scalar" type to be copied.
I think this is a good idea.  I doubt it's enough to fix all places that 
don't inherit from numpy scalars, but it's a start.

It seems like we need to figure out where the beginning of the data is 
for the type which is assumed to be defined on alignment boundaries 
after a PyObject_HEAD  (right)?  This could actually be used for 
everything and all the switch and if statements eliminated.

I think the alignment field is the only thing needed, though.  I don't 
see how I would use the elsize field?
>
> As it stands, if the user-defined type does not inherit from a Numpy 
> scalar, lots of things continue to break.
Not surprising, I did not make sure and support this.

> Furthermore it seems like the scalar conversions prefer the builtin 
> types, but it seems to me that the user-defined type should be preferred.
I'm not sure what this means. 
>
>
> i.e. if I try to get an element from my mxDateTime array, I get a 
> float back:
> >>> mxArr[0] = DateTime.now()
> >>> mxArr[0][0]
> 732610.60691268521
Why can you index mxArr[0]?  What is mxArr[0]?  If it's a scalar, then 
why can you index it?  What is type(mxArr[0])?

>
> But what I really want is the mxDateTime, which, oddly enough, is what 
> happens if I use tolist():
> >>> mxArr.tolist()[0]
> [<DateTime object for '2006-10-27 14:33:57.25' at a73c60>]

That's not surprising because tolist just calls getitem on each element 
in the array to construct the list.

-Travis


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642




More information about the Numpy-discussion mailing list