[SciPy-User] Populating a recarray from 0 size

Benjamin Root ben.root@ou....
Sun Aug 22 16:41:42 CDT 2010


On Sun, Aug 22, 2010 at 2:13 PM, Benjamin Root <ben.root@ou.edu> wrote:

> On Sun, Aug 22, 2010 at 1:46 PM, Joe Kington <jkington@wisc.edu> wrote:
>
>>
>>
>> On Sun, Aug 22, 2010 at 12:44 PM, Benjamin Root <ben.root@ou.edu> wrote:
>>
>>> On Sat, Aug 21, 2010 at 8:18 PM, Sergi Pons Freixes <spons@utm.csic.es>wrote:
>>>
>>>> Hi everybody,
>>>>
>>>> I'm interested in populating a recarray a row at a time. I thought in
>>>> using:
>>>>
>>>> -------
>>>> # Creation of empty recarray
>>>> names = ["cruise", "SEQ", "param", "r2", "stderr", "slope", "intercept"
>>>> formats = ["i4", "i4", "S10", "f4", "f4", "f4", "f4"]
>>>> statsc = scipy.empty(0, dtype={"names":names, "formats":formats})
>>>>
>>>> # Adding row and setting values
>>>> statsc = scipy.resize(statsc, statsc.size + 1)
>>>> statsc["cruise"][-1] = cr
>>>> statsc["SEQ"][-1] = ca
>>>> statsc["param"][-1] = yvar
>>>> ...
>>>> -------
>>>>
>>>> But scipy.resize complains with:
>>>> statsc = scipy.resize(statsc, statsc.size + 1)
>>>>  File "/usr/lib/python2.6/site-packages/numpy/core/fromnumeric.py",
>>>> line 833, in resize
>>>>    if not Na: return mu.zeros(new_shape, a.dtype.char)
>>>> ValueError: Empty data-type
>>>>
>>>> So, is scipy not happy because statsc.size is 0 on the first resize?
>>>> In this case, how could I overcome this limitation?
>>>>
>>>> Regards,
>>>> Sergi
>>>>
>>>>
>>> Sergi,
>>>
>>> I had a similar need recently.  I was populating recarrays where I didn't
>>> know the final size beforehand.  The way I solved the problem was to create
>>> an iterator function and use .fromiter() to build the array.
>>>
>>> A simplified example:
>>>
>>> import numpy
>>>
>>> class foobar(object) :
>>>     def __init__(self) :
>>>        self._myval = 0
>>>
>>>     def __iter__(self) :
>>>         return self
>>>
>>>     def next(self) :
>>>         if self._myval >= 7 :
>>>             raise StopIteration
>>>
>>>         self._myval += 1
>>>         return self._myval
>>>
>>> gen = foobar()
>>> a = numpy.fromiter(gen, dtype=[('Column', 'i4')])
>>> print a
>>>
>>>
>>> However... for some reason, I can't seem to get this working.  Maybe
>>> someone else can spot my error?
>>>
>>
>> I think the iterator needs to return a tuple, rather than a "bare" int.
>>  At any rate, if you change "return self._myval" to "return (self._myval,)",
>> things work perfectly.
>>
>
> Ah, right!
>
> An even easier example would be:
>
> def foobar(start, end) :
>     curVal = start
>     while curVal < end :
>         yield (curVal,)
>         curVal += 1
>
> gen = foo(1, 10)
> a = numpy.fromiter(gen, [('Column', 'i4')])
> print a['Column']
>
> Which woulld output:
>
> [1 2 3 4 5 6 7 8 9]
>
> I hope that helps!
> Ben Root
>
>
I should also add that the .fromiter() method is extremely flexible.  It is
perfect for those who don't know the number of elements ahead of time, but
it also works for those who do know how many elements they will be loading.
One can pass a "count" keyword to the .fromiter() to have the function
pre-allocate the array for you.

Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/scipy-user/attachments/20100822/118b423f/attachment.html 


More information about the SciPy-User mailing list