[Numpy-discussion] "Reference count error detected" bug appears with multithreading (OpenMP & TBB)

Robert Kern robert.kern@gmail....
Wed Jan 18 10:54:53 CST 2012


On Wed, Jan 18, 2012 at 16:14, Malcolm Reynolds
<malcolm.reynolds@gmail.com> wrote:
>>
>> I suspect that you are obtaining the numpy object (1 Py_INCREF) before
>> you split into multiple threads but releasing them in each thread
>> (multiple Py_DECREFs). This is probably being hidden from you by the
>> boost.python interface and/or the boost::detail::sp_counted_impl_p<>
>> smart(ish) pointer. Check the backtrace where your code starts to
>> verify if this looks to be the case.
>
> Thankyou for your quick reply. This makes a lot of sense, I'm just
> having trouble seeing where this could be happening as everything I
> pass into each parallel computation strand is pass down as either
> pointer-to-consts or reference-to-const - the only things that need to
> be modified (for example random number generator objects) are created
> uniquely inside each iteration of the for loop so it can't be that.

My C++-fu is fairly weak, so I'm never really sure what the smart
pointers are doing when. If there are tracing features that you can
turn on, try that. Is this deallocation of the smart pointer to the
"garf::multivariate_normal<double> const" being done inside the loop
or outside back in the main thread? Where did it get created?

> This information about which object has the reference count problem
> helps though, I will keep digging. I'm vaguely planning on trying to
> track every incref and decref so I can pin down which object has an
> unbalanced amount - to do this I want to know the address of the
> array, rather than the associated datatype descriptor - I assume I
> want to pay attention to the (self=0x117e0e850) in this line, and that
> is the address of the array I am mishandling?
>
> #1  0x0000000102897fc4 in array_dealloc (self=0x117e0e850) at arrayobject.c:271

Yes.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


More information about the NumPy-Discussion mailing list