[SciPy-User] Populating a recarray from 0 size
Sun Aug 22 13:52:09 CDT 2010
On Sun, Aug 22, 2010 at 2:23 PM, Sergi Pons Freixes <firstname.lastname@example.org> wrote:
> On Sat, Aug 21, 2010 at 6:29 PM, <email@example.com> wrote:
>> BTW: These are numpy functions.
> Oh, sorry, I am so used to call numpy functions via scipy that I
> forgot about it.
>> I don't know the answer, but it would be more efficient to correctly
>> size the empty array, if you know the size, or to build up the array
>> first as a list of tuples before converting to an array.
> The problem is that the size is not known. I'll think about the list of tuples.
> On Sun, Aug 22, 2010 at 8:19 AM, Åsmund Hjulstad
> <firstname.lastname@example.org> wrote:
>> My approach in this kind of situation has been using pytables, creating an
>> empty table with a specific dtype, and then populating the table with the
>> append method. It seems very efficient, and quite convenient in situations
>> where I eventually will need to persist data to disk somehow.
>> Reading the table into an in memory recarray is then easily done using
>> data = h5file.root.datatable[:].view(numpy.recarray)
> Mmm, it sounds a bit overkill to use pytables just for having a
> recarray with dynamic size. Anyway, I'll have a look at it, thank you.
> I would also appreciate is a reasoning of why is scipy.resize (e.g.,
> numpy.resize) raising this exception. The idea increasing the size of
> a 0-size array doesn't sound very weird to me, so I was shocked that i
> wasn't working. Actually, I've been successfully using the method
> array.resize(newsize) previously, but now I'm "forced" to use
> scipy.resize because of the "ValueError: cannot resize an array that
> has been referenced or is referencing another array in this way."
> issue (see ).
I think it should be a bug report on the numpy trac, given that the
method works but not the function.
To use the method, you might also be able to make a copy yourself to
break the second reference. (Not tried out).
I still think copying the entire array whenever a new item is added is
not a good idea (very costly in a loop).
> If this conversation is more appropriate to be developed in the numpy
> mailing list, please let me now and I will migrate it.
>  http://www.scipy.org/Numpy_Example_List#head-d5585278b02f43f78eb75a4fdc337fdc12417426
> SciPy-User mailing list
More information about the SciPy-User