[Numpy-discussion] Segfault in PyArray_Item_XDECREF when using recarray object references titles

Michael Droettboom mdroe@stsci....
Tue Jul 22 14:54:38 CDT 2008


I also noticed that the inverse operation, PyArray_Item_INCREF has the 
potential to leak memory as it will doubly-increment each object in the 
array.  The solution there probably isn't quite as clean, since we can't 
just mark the pointer.  It will have to somehow avoid incref'ing the 
objects twice when iterating through the fields dictionary.

Cheers,
Mike

Michael Droettboom wrote:
> I've run into a segfault that occurs in the array destructor with 
> arrays containing object references with both names and titles.
>
> When a field contains both and name and a title, the fields dictionary 
> contains two entries for that field.  This means that the array item 
> destructor (which iterates through the fields dictionary) will decref 
> the pointed-to object twice.  If the first decref causes the object to 
> be deleted, the second decref has the potential to segfault.
>
> It seems the simplest patch is to set the object pointer to NULL after 
> decref'ing, so the second decref will do nothing.  However, perhaps 
> there is a way to avoid decref'ing twice in the first place.
>
> I've attached a script that exercises the segfault, a gdb backtrace, 
> and a patch.  You may need to adjust the number of rows until it is 
> high enough to create a segfault on your system.
>
> This is on:
> RHEL4
> Python 2.5.2
> Numpy SVN r5497
>
> Cheers,
> Mike
>
>> gdb python
> GNU gdb Red Hat Linux (6.3.0.0-1.153.el4_6.2rh)
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and 
> you are
> welcome to change it and/or distribute copies of it under certain 
> conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for 
> details.
> This GDB was configured as "i386-redhat-linux-gnu"...Using host 
> libthread_db library "/lib/tls/libthread_db.so.1".
>
> (gdb) run segfault.py
> Starting program: /wonkabar/data1/usr/bin/python segfault.py
> [Thread debugging using libthread_db enabled]
> [New Thread -1208489312 (LWP 30028)]
> len(dtype) = 1, len(dtype.fields) = 2
> {'name': (dtype('object'), 0, 'title'), 'title': (dtype('object'), 0, 
> 'title')}
>
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread -1208489312 (LWP 30028)]
> 0x0097285e in PyArray_Item_XDECREF (
>    data=0xb7a3e780 "\uffff_\224\uffff 
> `\214\uffff(`\214\uffff0`\214\uffff8`\214\uffff@`\214\uffffH`\214\uffffP`\214\uffffX`\214\uffff``\214\uffffh`\214\uffffp`\214\uffffx`\214\uffff\200`\214\uffff\210`\214\uffff\220`\214\uffff\230`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff", 
>
>    descr=0x9d4680) at numpy/core/src/arrayobject.c:198
> 198             Py_XDECREF(*temp);
> (gdb) bt
> #0  0x0097285e in PyArray_Item_XDECREF (
>    data=0xb7a3e780 "\uffff_\224\uffff 
> `\214\uffff(`\214\uffff0`\214\uffff8`\214\uffff@`\214\uffffH`\214\uffffP`\214\uffffX`\214\uffff``\214\uffffh`\214\uffffp`\214\uffffx`\214\uffff\200`\214\uffff\210`\214\uffff\220`\214\uffff\230`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff", 
>
>    descr=0x9d4680) at numpy/core/src/arrayobject.c:198
> #1  0x00991bc7 in PyArray_XDECREF (mp=0xb7ae4f0c)
>    at numpy/core/src/arrayobject.c:211
> #2  0x009a579b in array_dealloc (self=0xb7ae4f0c)
>    at numpy/core/src/arrayobject.c:2089
> #3  0x0809781f in subtype_dealloc (self=0xb7ae4f0c) at 
> Objects/typeobject.c:709
> #4  0x08082a02 in PyDict_SetItem (op=0xb7f56acc, key=0xb7ea7d80,
>    value=0x81379c0) at Objects/dictobject.c:416
> #5  0x08085a1e in _PyModule_Clear (m=0xb7f3e0ec) at 
> Objects/moduleobject.c:136
> #6  0x080d7138 in PyImport_Cleanup () at Python/import.c:439
> #7  0x080e4343 in Py_Finalize () at Python/pythonrun.c:399
> #8  0x08056633 in Py_Main (argc=1, argv=0xbff1ca24) at Modules/main.c:545
> #9  0x08056323 in main (argc=2, argv=0xbff1ca24) at ./Modules/python.c:23
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion

-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA



More information about the Numpy-discussion mailing list