[Numpy-discussion] subclassing array in c

Christoph Gohle christoph.gohle@mpq.mpg...
Thu Mar 8 01:49:18 CST 2012


Dear Val,

I agree that more detail is needed. Sorry for that it was late yesterday.

 I am running Python 2.6.1, numpy development branch (numpy-2.0.0.dev_20101104-py2.6-macosx-10.6-universal.egg). maybe I should switch to release?

I compile with your setup.py using 'python setup.py build_ext -i' and run the commands below in ipython. As you can see, I don't get a crash for a single call to the constructor, consistent with your observation. And it looks like, one has to use dict in the unit to make it crash.

Can you make sense out of that?


In [1]: import spampub

In [4]: spampub.UnitArray(1,{'s':1})
Out[4]: UnitArray(1)

In [6]: a=[spampub.UnitArray(i,{'s':i}) for i in xrange(1000)]
Segmentation fault

backtrace is the following:

Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000021
Crashed Thread:  0  Dispatch queue: com.apple.main-thread

Thread 0 Crashed:  Dispatch queue: com.apple.main-thread
0   org.python.python             	0x00000001000b0b8e PyObject_GC_Track + 1473
1   org.python.python             	0x00000001000b1255 _PyObject_GC_Malloc + 191
2   org.python.python             	0x00000001000b12d0 _PyObject_GC_New + 21
3   org.python.python             	0x000000010003a856 PyDict_New + 116
4   org.python.python             	0x000000010003a99c _PyDict_NewPresized + 24
5   org.python.python             	0x00000001000880cb PyEval_EvalFrameEx + 11033
6   org.python.python             	0x000000010008acce PyEval_EvalCodeEx + 1803
7   org.python.python             	0x000000010008735d PyEval_EvalFrameEx + 7595
8   org.python.python             	0x000000010008acce PyEval_EvalCodeEx + 1803
9   org.python.python             	0x000000010008935e PyEval_EvalFrameEx + 15788
10  org.python.python             	0x000000010008acce PyEval_EvalCodeEx + 1803
11  org.python.python             	0x000000010008935e PyEval_EvalFrameEx + 15788
12  org.python.python             	0x00000001000892e1 PyEval_EvalFrameEx + 15663
13  org.python.python             	0x000000010008acce PyEval_EvalCodeEx + 1803
14  org.python.python             	0x000000010008935e PyEval_EvalFrameEx + 15788
15  org.python.python             	0x000000010008acce PyEval_EvalCodeEx + 1803
16  org.python.python             	0x000000010008935e PyEval_EvalFrameEx + 15788
17  org.python.python             	0x000000010008acce PyEval_EvalCodeEx + 1803
18  org.python.python             	0x000000010008935e PyEval_EvalFrameEx + 15788
19  org.python.python             	0x000000010008acce PyEval_EvalCodeEx + 1803
20  org.python.python             	0x000000010008935e PyEval_EvalFrameEx + 15788
21  org.python.python             	0x000000010008acce PyEval_EvalCodeEx + 1803
22  org.python.python             	0x000000010008ad61 PyEval_EvalCode + 54
23  org.python.python             	0x00000001000a265a Py_CompileString + 78
24  org.python.python             	0x00000001000a2723 PyRun_FileExFlags + 150
25  org.python.python             	0x00000001000a423d PyRun_SimpleFileExFlags + 704
26  org.python.python             	0x00000001000b0286 Py_Main + 2718
27  org.python.python.app         	0x0000000100000e6c start + 52

Am 08.03.2012 um 02:36 schrieb Val Kalatsky:

> 
> Tried it on my Ubuntu 10.10 box, no problem:
> 
> 1) Saved as spampub.c
> 2) Compiled with (setup.py attached): python setup.py build_ext -i
> 3) Tested from ipython:
> In [1]: import spampub
> In [2]: ua=spampub.UnitArray([0,1,2,3.0],'liter')
> In [3]: ua
> Out[3]: UnitArray([ 0.,  1.,  2.,  3.])
> In [4]: ua.unit
> Out[4]: 'liter'
> 
> 
> On Wed, Mar 7, 2012 at 7:15 PM, Val Kalatsky <kalatsky@gmail.com> wrote:
> 
> Seeing the backtrace would be helpful. 
> Can you do whatever leads to the segfault 
> from python run from gdb?
> Val
> 
> 
> On Wed, Mar 7, 2012 at 7:04 PM, Christoph Gohle <christoph.gohle@mpq.mpg.de> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi,
> 
> I have been struggeling for quite some time now. Desperate as I am, now I need help.
> 
> I was trying to subclass ndarrays in a c extension (see code below) and do constantly get segfaults. I have been checking my INCREF and DECREF stuff up and down but can't find the error. Probably I got something completely wrong... anybody able to help?
> 
> Thanks,
> Christoph
> - -----------------
> #include <Python.h>
> #include <structmember.h>
> 
> #include <numpy/arrayobject.h>
> 
> static PyObject *SpamError;
> 
> typedef struct {
>  PyArrayObject base;
>  PyDictObject* unit;
> } UnitArrayObject;
> 
> PyTypeObject unitArrayObjectType;
> 
> static int
> checkunit(PyObject *unit1, PyObject *unit2) {
>  return PyObject_Compare(unit1, unit2);
> }
> 
> static PyObject *
> unitArray_new(PyTypeObject *cls, PyObject *args, PyObject *kwargs) {
>        PyObject *data = NULL,
>                         *unit = NULL;
>  PyArray_Descr* dtype = NULL;
>  PyObject *res = NULL, *tmp = NULL;
> 
>        if (!PyArg_ParseTuple(args, "OO|O&", &data, &unit, PyArray_DescrConverter, &dtype)) {
>                Py_XDECREF(dtype);
>                return NULL;
>        }
> 
>  res = PyArray_FromAny(data, dtype, 0, 0, NPY_ENSURECOPY, NULL);
>        if (res == NULL) {
>                Py_XDECREF(dtype);
>    //TODO: raise exception?
>                return NULL;
>        }
> 
>        if (PyObject_IsInstance(data, (PyObject*)cls)) {
>                if (unit!=NULL && !checkunit((PyObject*)((UnitArrayObject*)data)->unit,unit)) {
>                        Py_XDECREF(res);
>                        //TODO: raise exception
>                        return NULL;
>                }
>        } else {
>                if (PyObject_IsTrue(unit)) {
>      tmp = res;
>                        res = PyArray_View((PyArrayObject*)res, NULL, &unitArrayObjectType);
>      if (tmp!=res) {
>        Py_XDECREF(tmp);
>      }
>      ((UnitArrayObject*)res)->unit = (PyDictObject*)unit;
>      Py_INCREF(unit);
>      if (unit!=NULL) {
>      }
>                }
>        }
>        return res;
> }
> 
> static PyObject*
> unitArray__array_finalize__(PyObject* new, PyObject* args) {
>        PyObject *attr = NULL, *tmp = NULL;
>  PyObject *parent = NULL;
> 
>  if (!PyArg_ParseTuple(args, "O", &parent)) {
>    return NULL;
>  }
>   if (parent!=NULL) {
>     attr = PyObject_GetAttrString(parent, "unit");
>     if (attr == NULL) {
>        //parent has no 'unit' so we make a new empty one
>       attr = PyDict_New();
>       PyErr_Clear();
>     }
>   }
>  tmp = (PyObject*)((UnitArrayObject*)new)->unit;
>    ((UnitArrayObject*)new)->unit = (PyDictObject*)attr;
> 
>  Py_INCREF(Py_None);
>  return Py_None;
> }
> 
> static PyObject*
> unitArray__array_wrap__(PyObject *self, PyObject *args) {
>        PyObject *array = NULL, *context = NULL;
> 
>        if (!PyArg_ParseTuple(args, "OO", array, context)) {
>                //TODO: raise exception
>                return NULL;
>        }
> 
>        printf("%s",PyString_AsString(PyObject_Str(context)));
> 
>  Py_INCREF(array);
>  return array;
> }
> 
> 
> static PyMethodDef unitArrayMethods[] = {
>  {"__array_finalize__", unitArray__array_finalize__, METH_VARARGS, "array finalize method"},
>  {"__array_wrap__", unitArray__array_wrap__, METH_VARARGS, "array wrap method"},
>  {NULL, NULL, 0, NULL}
> };
> 
> static PyMemberDef unitArrayMembers[] = {
>  {"unit", T_OBJECT, offsetof(UnitArrayObject, unit),  0, "dictionary containing unit info."},
>  {NULL, 0, 0, 0, NULL}
> };
> 
> PyTypeObject unitArrayObjectType = {
>        PyObject_HEAD_INIT(NULL)
>        0,                              /* ob_size        */
>        "spam.UnitArray",               /* tp_name        */
>        sizeof(UnitArrayObject),                /* tp_basicsize   */
>        0,                              /* tp_itemsize    */
>        0,                              /* tp_dealloc     */
>        0,                              /* tp_print       */
>        0,                              /* tp_getattr     */
>        0,                              /* tp_setattr     */
>        0,                              /* tp_compare     */
>        0,                              /* tp_repr        */
>        0,                              /* tp_as_number   */
>        0,                              /* tp_as_sequence */
>        0,                              /* tp_as_mapping  */
>        0,                              /* tp_hash        */
>        0,                              /* tp_call        */
>        0,                              /* tp_str         */
>        0,                              /* tp_getattro    */
>        0,                              /* tp_setattro    */
>        0,                              /* tp_as_buffer   */
>        Py_TPFLAGS_DEFAULT,             /* tp_flags       */
>        "A numpy array with units",     /* tp_doc         */
>  0,        /* traverseproc */
>  0,        /* tp_clear*/
>  0,        /* tp_richcompare */
>  0,        /* tp_weaklistoffset */
>  0,        /* tp_iter */
>  0,        /* tp_iternext */
>  unitArrayMethods,        /* tp_methods */
>  unitArrayMembers,        /* tp_members */
>  0,        /* tp_getset */
>  0,        /* tp_base*/
>  0,        /* tp_dict */
>  0,        /* tp_descr_get*/
>  0,        /* tp_descr_set */
>  0,        /* tp_dictoffset */
>  0,        /* tp_init */
>  0,        /* tp_alloc */
>  unitArray_new /* tp_new */
> };
> 
> 
> static PyMethodDef SpamMethods[] = {
>    {NULL, NULL, 0, NULL}        /* Sentinel */
> };
> 
> PyObject *unitArray = NULL;
> PyMODINIT_FUNC
> initspampub(void)
> {
>  import_array();
> 
>  PyObject *m;
> 
>  Py_INCREF(&PyArray_Type);
>  unitArrayObjectType.tp_base = &PyArray_Type;
> 
>  if (PyType_Ready(&unitArrayObjectType) < 0)
>    return;
> 
> 
>  m = Py_InitModule3("spampub", SpamMethods, "some tests and a array type with units.");
>  if (m == NULL)
>    return;
> 
>  SpamError = PyErr_NewException("spampub.error", NULL, NULL);
>  Py_INCREF(SpamError);
>  PyModule_AddObject(m, "error", SpamError);
>  Py_INCREF(&unitArrayObjectType);
>  PyModule_AddObject(m, "UnitArray", (PyObject *)&unitArrayObjectType);
>  (void) Py_InitModule("spampub", SpamMethods);
> 
> }
> 
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
> 
> iEYEARECAAYFAk9YBbkACgkQLYu25rCEIzsExQCggHg0e0ibYP3bEsAE0Ce8Rbm3
> qOcAoKwuTuE4BDZgrqNpwISgMS6uSMnO
> =Qtek
> -----END PGP SIGNATURE-----
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> 
> <setup.py>_______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion


Christoph Gohle
--
Max-Planck-Institut für Quantenoptik
Abteilung Quantenvielteilchensysteme
Hans-Kopfermann-Strasse 1
85748 Garching

christoph.gohle@mpq.mpg.de
tel: +49 89 32905 283
fax: +49 89 32905 313



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20120308/4a95ce0e/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 203 bytes
Desc: Signierter Teil der Nachricht
Url : http://mail.scipy.org/pipermail/numpy-discussion/attachments/20120308/4a95ce0e/attachment-0001.bin 


More information about the NumPy-Discussion mailing list