[Numpy-discussion] Alternative to record array

Jim Vickroy Jim.Vickroy@noaa....
Mon Dec 29 11:37:21 CST 2008


Jean-Baptiste Rudant wrote:
> Hello,
>
> I like to use record arrays to access fields by their name, and 
> because they are esay to use with pytables. But I think it's not very 
> effiicient for what I have to do. Maybe I'm misunderstanding something.
>
> Example : 
>
> import numpy as np
> age = np.random.randint(0, 99, 10e6)
> weight = np.random.randint(0, 200, 10e6)
> data = np.rec.fromarrays((age, weight), names='age, weight')
> # the kind of operations I do is :
> data.age += data.age + 1
> # but it's far less efficient than doing :
> age += 1
> # because I think the record array stores [(age_0, weight_0) 
> ...(age_n, weight_n)]
> # and not [age0 ... age_n] then [weight_0 ... weight_n].
Sorry I am not able to answer your question; I am really a new user of 
numpy also.

It does seem the addition operation is more than 4 times slower, when 
using record arrays, based on the following:

 >>> import numpy, sys, timeit
 >>> sys.version
'2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)]'
 >>> numpy.__version__
'1.2.1'
 >>> count = 10e6
 >>> ages  = numpy.random.randint(0,100,count)
 >>> weights = numpy.random.randint(1,200,count)
 >>> data = numpy.rec.fromarrays((ages,weights),names='ages,weights')
 >>>
 >>> timer = timeit.Timer('data.ages += 1','from __main__ import data')
 >>> timer.timeit(number=100)
30.110649537860262
 >>>
 >>> timer = timeit.Timer('ages += 1','from __main__ import ages')
 >>> timer.timeit(number=100)
6.9850710076280507
 >>>
>
> So I think I don't use record arrays for the right purpose. I only 
> need something which would make me esasy to manipulate data by 
> accessing fields by their name.
>
> Am I wrong ? Is their something in numpy for my purpose ? Do I have to 
> implement my own class, with something like :
>
>
> class FieldArray:
>     def __init__(self, array_dict):
>         self.array_list = array_dict
>             
>     def __getitem__(self, field):
>         return self.array_list[field]
>     
>     def __setitem__(self, field, value):
>         self.array_list[field] = value
>     
> my_arrays = {'age': age, 'weight' : weight}
> data = FieldArray(my_arrays)
>
> data['age'] += 1
>
> Thank you for the help,
>
> Jean-Baptiste Rudant
>
>
>
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>   

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://projects.scipy.org/pipermail/numpy-discussion/attachments/20081229/6ea27eec/attachment.html 


More information about the Numpy-discussion mailing list