[Numpy-discussion] fromiter and objects

[Travis Oliphant]

Tim Hochberg wrote:

>things are often not as easy as they appear, the real reason I'm writing 
>is this comment that explains why object arrays are disallowed in 
>fromiter (multiarraymodule.c::PyArray_:FromIter)
>
>            /* We would need to alter the memory RENEW code to decrement any
>               reference counts before just throwing away the memory.
>             */
>
>This doesn't seem right. The array that we would be RENEWing is a bunch 
>of PyObject*s. The reference counts don't reside there, but in the 
>objects themselves. When we do the RENEW, we don't want the reference 
>counts to change at all. 
>
I'm not sure if you are still thinking about this.  I suspect I added 
the comment because I was concerned about two things:

1) a resize that deletes memory without first decrefing object counts 
--- I suspose this can't happen here, though, because RENEW is always 
"growing" memory, right?

2) a resize that grows memory without inserting NULLS.  For Object array 
manipulation to work, you need initialized (to NULL) "empty" memory. 

>The one tricky case is if we run out of memory, 
>I'm not certain that the current setup correctly deals with reference 
>counts in this case, although it appears likely that it should work 
>since ret->data should still point to a valid chunk of memory and 
>decreffing ret should result in the subsequent deallocation of all the 
>  
>
>stored objects.
>  
>
As long as you don't store the pointer to new memory until after the 
RENEW was successful then this should work.

I think my comment came from the possibility of #1 in general use of 
RENEW, but I don't think that is going to happen.  Therefore, the only 
tweak that seams necessary is to initialize any new memory to NULL for 
OBJECT arrays.

-Travis