[Numpy-discussion] subclassing array in c

Val Kalatsky kalatsky@gmail....
Thu Mar 8 02:08:19 CST 2012


Hi Christoph,

I've just tried
a=[spampub.UnitArray(i,{'s':i}) for i in xrange(1000)]
and everything looks fine on my side.
Probably my test environment is too different to give comparable results:

In [3]: call(["uname", "-a"])
Linux ubuntu 2.6.35-22-generic #33-Ubuntu SMP Sun Sep 19 20:32:27 UTC 2010
x86_64 GNU/Linux
In [4]: np.__version__
Out[4]: '1.5.1'

I am afraid I will not be able to help testing it on super-nova versions of
numpy.
Cheers
Val


On Thu, Mar 8, 2012 at 1:49 AM, Christoph Gohle
<christoph.gohle@mpq.mpg.de>wrote:

> Dear Val,
>
> I agree that more detail is needed. Sorry for that it was late yesterday.
>
>  I am running Python 2.6.1, numpy development branch
> (numpy-2.0.0.dev_20101104-py2.6-macosx-10.6-universal.egg). maybe I should
> switch to release?
>
> I compile with your setup.py using 'python setup.py build_ext -i' and run
> the commands below in ipython. As you can see, I don't get a crash for a
> single call to the constructor, consistent with your observation. And it
> looks like, one has to use dict in the unit to make it crash.
>
> Can you make sense out of that?
>
>
> In [1]: import spampub
>
> In [4]: spampub.UnitArray(1,{'s':1})
> Out[4]: UnitArray(1)
>
> In [6]: a=[spampub.UnitArray(i,{'s':i}) for i in xrange(1000)]
> Segmentation fault
>
> backtrace is the following:
>
> Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
> Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000021
> Crashed Thread:  0  Dispatch queue: com.apple.main-thread
>
> Thread 0 Crashed:  Dispatch queue: com.apple.main-thread
> 0   org.python.python             0x00000001000b0b8e PyObject_GC_Track +
> 1473
> 1   org.python.python             0x00000001000b1255 _PyObject_GC_Malloc
> + 191
> 2   org.python.python             0x00000001000b12d0 _PyObject_GC_New + 21
> 3   org.python.python             0x000000010003a856 PyDict_New + 116
> 4   org.python.python             0x000000010003a99c _PyDict_NewPresized
> + 24
> 5   org.python.python             0x00000001000880cb PyEval_EvalFrameEx +
> 11033
> 6   org.python.python             0x000000010008acce PyEval_EvalCodeEx +
> 1803
> 7   org.python.python             0x000000010008735d PyEval_EvalFrameEx +
> 7595
> 8   org.python.python             0x000000010008acce PyEval_EvalCodeEx +
> 1803
> 9   org.python.python             0x000000010008935e PyEval_EvalFrameEx +
> 15788
> 10  org.python.python             0x000000010008acce PyEval_EvalCodeEx +
> 1803
> 11  org.python.python             0x000000010008935e PyEval_EvalFrameEx +
> 15788
> 12  org.python.python             0x00000001000892e1 PyEval_EvalFrameEx +
> 15663
> 13  org.python.python             0x000000010008acce PyEval_EvalCodeEx +
> 1803
> 14  org.python.python             0x000000010008935e PyEval_EvalFrameEx +
> 15788
> 15  org.python.python             0x000000010008acce PyEval_EvalCodeEx +
> 1803
> 16  org.python.python             0x000000010008935e PyEval_EvalFrameEx +
> 15788
> 17  org.python.python             0x000000010008acce PyEval_EvalCodeEx +
> 1803
> 18  org.python.python             0x000000010008935e PyEval_EvalFrameEx +
> 15788
> 19  org.python.python             0x000000010008acce PyEval_EvalCodeEx +
> 1803
> 20  org.python.python             0x000000010008935e PyEval_EvalFrameEx +
> 15788
> 21  org.python.python             0x000000010008acce PyEval_EvalCodeEx +
> 1803
> 22  org.python.python             0x000000010008ad61 PyEval_EvalCode + 54
> 23  org.python.python             0x00000001000a265a Py_CompileString + 78
> 24  org.python.python             0x00000001000a2723 PyRun_FileExFlags +
> 150
> 25  org.python.python             0x00000001000a423d
> PyRun_SimpleFileExFlags + 704
> 26  org.python.python             0x00000001000b0286 Py_Main + 2718
> 27  org.python.python.app         0x0000000100000e6c start + 52
>
> Am 08.03.2012 um 02:36 schrieb Val Kalatsky:
>
>
> Tried it on my Ubuntu 10.10 box, no problem:
>
> 1) Saved as spampub.c
> 2) Compiled with (setup.py attached): python setup.py build_ext -i
> 3) Tested from ipython:
> In [1]: import spampub
> In [2]: ua=spampub.UnitArray([0,1,2,3.0],'liter')
> In [3]: ua
> Out[3]: UnitArray([ 0.,  1.,  2.,  3.])
> In [4]: ua.unit
> Out[4]: 'liter'
>
>
> On Wed, Mar 7, 2012 at 7:15 PM, Val Kalatsky <kalatsky@gmail.com> wrote:
>
>>
>> Seeing the backtrace would be helpful.
>> Can you do whatever leads to the segfault
>> from python run from gdb?
>> Val
>>
>>
>> On Wed, Mar 7, 2012 at 7:04 PM, Christoph Gohle <
>> christoph.gohle@mpq.mpg.de> wrote:
>>
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>>
>>> Hi,
>>>
>>> I have been struggeling for quite some time now. Desperate as I am, now
>>> I need help.
>>>
>>> I was trying to subclass ndarrays in a c extension (see code below) and
>>> do constantly get segfaults. I have been checking my INCREF and DECREF
>>> stuff up and down but can't find the error. Probably I got something
>>> completely wrong... anybody able to help?
>>>
>>> Thanks,
>>> Christoph
>>> - -----------------
>>> #include <Python.h>
>>> #include <structmember.h>
>>>
>>> #include <numpy/arrayobject.h>
>>>
>>> static PyObject *SpamError;
>>>
>>> typedef struct {
>>>  PyArrayObject base;
>>>  PyDictObject* unit;
>>> } UnitArrayObject;
>>>
>>> PyTypeObject unitArrayObjectType;
>>>
>>> static int
>>> checkunit(PyObject *unit1, PyObject *unit2) {
>>>  return PyObject_Compare(unit1, unit2);
>>> }
>>>
>>> static PyObject *
>>> unitArray_new(PyTypeObject *cls, PyObject *args, PyObject *kwargs) {
>>>        PyObject *data = NULL,
>>>                         *unit = NULL;
>>>  PyArray_Descr* dtype = NULL;
>>>  PyObject *res = NULL, *tmp = NULL;
>>>
>>>        if (!PyArg_ParseTuple(args, "OO|O&", &data, &unit,
>>> PyArray_DescrConverter, &dtype)) {
>>>                Py_XDECREF(dtype);
>>>                return NULL;
>>>        }
>>>
>>>  res = PyArray_FromAny(data, dtype, 0, 0, NPY_ENSURECOPY, NULL);
>>>        if (res == NULL) {
>>>                Py_XDECREF(dtype);
>>>    //TODO: raise exception?
>>>                return NULL;
>>>        }
>>>
>>>        if (PyObject_IsInstance(data, (PyObject*)cls)) {
>>>                if (unit!=NULL &&
>>> !checkunit((PyObject*)((UnitArrayObject*)data)->unit,unit)) {
>>>                        Py_XDECREF(res);
>>>                        //TODO: raise exception
>>>                        return NULL;
>>>                }
>>>        } else {
>>>                if (PyObject_IsTrue(unit)) {
>>>      tmp = res;
>>>                        res = PyArray_View((PyArrayObject*)res, NULL,
>>> &unitArrayObjectType);
>>>      if (tmp!=res) {
>>>        Py_XDECREF(tmp);
>>>      }
>>>      ((UnitArrayObject*)res)->unit = (PyDictObject*)unit;
>>>      Py_INCREF(unit);
>>>      if (unit!=NULL) {
>>>      }
>>>                }
>>>        }
>>>        return res;
>>> }
>>>
>>> static PyObject*
>>> unitArray__array_finalize__(PyObject* new, PyObject* args) {
>>>        PyObject *attr = NULL, *tmp = NULL;
>>>  PyObject *parent = NULL;
>>>
>>>  if (!PyArg_ParseTuple(args, "O", &parent)) {
>>>    return NULL;
>>>  }
>>>   if (parent!=NULL) {
>>>     attr = PyObject_GetAttrString(parent, "unit");
>>>     if (attr == NULL) {
>>>        //parent has no 'unit' so we make a new empty one
>>>       attr = PyDict_New();
>>>       PyErr_Clear();
>>>     }
>>>   }
>>>  tmp = (PyObject*)((UnitArrayObject*)new)->unit;
>>>    ((UnitArrayObject*)new)->unit = (PyDictObject*)attr;
>>>
>>>  Py_INCREF(Py_None);
>>>  return Py_None;
>>> }
>>>
>>> static PyObject*
>>> unitArray__array_wrap__(PyObject *self, PyObject *args) {
>>>        PyObject *array = NULL, *context = NULL;
>>>
>>>        if (!PyArg_ParseTuple(args, "OO", array, context)) {
>>>                //TODO: raise exception
>>>                return NULL;
>>>        }
>>>
>>>        printf("%s",PyString_AsString(PyObject_Str(context)));
>>>
>>>  Py_INCREF(array);
>>>  return array;
>>> }
>>>
>>>
>>> static PyMethodDef unitArrayMethods[] = {
>>>  {"__array_finalize__", unitArray__array_finalize__, METH_VARARGS,
>>> "array finalize method"},
>>>  {"__array_wrap__", unitArray__array_wrap__, METH_VARARGS, "array wrap
>>> method"},
>>>  {NULL, NULL, 0, NULL}
>>> };
>>>
>>> static PyMemberDef unitArrayMembers[] = {
>>>  {"unit", T_OBJECT, offsetof(UnitArrayObject, unit),  0, "dictionary
>>> containing unit info."},
>>>  {NULL, 0, 0, 0, NULL}
>>> };
>>>
>>> PyTypeObject unitArrayObjectType = {
>>>        PyObject_HEAD_INIT(NULL)
>>>        0,                              /* ob_size        */
>>>        "spam.UnitArray",               /* tp_name        */
>>>        sizeof(UnitArrayObject),                /* tp_basicsize   */
>>>        0,                              /* tp_itemsize    */
>>>        0,                              /* tp_dealloc     */
>>>        0,                              /* tp_print       */
>>>        0,                              /* tp_getattr     */
>>>        0,                              /* tp_setattr     */
>>>        0,                              /* tp_compare     */
>>>        0,                              /* tp_repr        */
>>>        0,                              /* tp_as_number   */
>>>        0,                              /* tp_as_sequence */
>>>        0,                              /* tp_as_mapping  */
>>>        0,                              /* tp_hash        */
>>>        0,                              /* tp_call        */
>>>        0,                              /* tp_str         */
>>>        0,                              /* tp_getattro    */
>>>        0,                              /* tp_setattro    */
>>>        0,                              /* tp_as_buffer   */
>>>        Py_TPFLAGS_DEFAULT,             /* tp_flags       */
>>>        "A numpy array with units",     /* tp_doc         */
>>>  0,        /* traverseproc */
>>>  0,        /* tp_clear*/
>>>  0,        /* tp_richcompare */
>>>  0,        /* tp_weaklistoffset */
>>>  0,        /* tp_iter */
>>>  0,        /* tp_iternext */
>>>  unitArrayMethods,        /* tp_methods */
>>>  unitArrayMembers,        /* tp_members */
>>>  0,        /* tp_getset */
>>>  0,        /* tp_base*/
>>>  0,        /* tp_dict */
>>>  0,        /* tp_descr_get*/
>>>  0,        /* tp_descr_set */
>>>  0,        /* tp_dictoffset */
>>>  0,        /* tp_init */
>>>  0,        /* tp_alloc */
>>>  unitArray_new /* tp_new */
>>> };
>>>
>>>
>>> static PyMethodDef SpamMethods[] = {
>>>    {NULL, NULL, 0, NULL}        /* Sentinel */
>>> };
>>>
>>> PyObject *unitArray = NULL;
>>> PyMODINIT_FUNC
>>> initspampub(void)
>>> {
>>>  import_array();
>>>
>>>  PyObject *m;
>>>
>>>  Py_INCREF(&PyArray_Type);
>>>  unitArrayObjectType.tp_base = &PyArray_Type;
>>>
>>>  if (PyType_Ready(&unitArrayObjectType) < 0)
>>>    return;
>>>
>>>
>>>  m = Py_InitModule3("spampub", SpamMethods, "some tests and a array type
>>> with units.");
>>>  if (m == NULL)
>>>    return;
>>>
>>>  SpamError = PyErr_NewException("spampub.error", NULL, NULL);
>>>  Py_INCREF(SpamError);
>>>  PyModule_AddObject(m, "error", SpamError);
>>>  Py_INCREF(&unitArrayObjectType);
>>>  PyModule_AddObject(m, "UnitArray", (PyObject *)&unitArrayObjectType);
>>>  (void) Py_InitModule("spampub", SpamMethods);
>>>
>>> }
>>>
>>>
>>> -----BEGIN PGP SIGNATURE-----
>>> Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
>>>
>>> iEYEARECAAYFAk9YBbkACgkQLYu25rCEIzsExQCggHg0e0ibYP3bEsAE0Ce8Rbm3
>>> qOcAoKwuTuE4BDZgrqNpwISgMS6uSMnO
>>> =Qtek
>>> -----END PGP SIGNATURE-----
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion@scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>
>>
> <setup.py>_______________________________________________
>
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
> Christoph Gohle
> --
> Max-Planck-Institut für Quantenoptik
> Abteilung Quantenvielteilchensysteme
> Hans-Kopfermann-Strasse 1
> 85748 Garching
>
> christoph.gohle@mpq.mpg.de
> tel: +49 89 32905 283
> fax: +49 89 32905 313
>
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20120308/e935714a/attachment-0001.html 


More information about the NumPy-Discussion mailing list