[SciPy-user] Efficient update of dictionary holding arrays

Ryan May rmay31@gmail....
Fri Jun 5 09:35:38 CDT 2009


2009/6/5 Jose Gómez-Dans <jgomezdans@gmail.com>

> Hi,
> This is probably something that is very python-centric and not very
> scipy-centric, but maybe the folk here can shed light on it. I am storing
> numpy arrays in dictionaries. The keys to such dicionaries are tuples, and
> hence fairly complex. The dictionaries are huge (they are around 100000
> elements, but potentially I'd like to process around 6M elements). Each
> dictionary element, in itself, points to another dictionary, with some 10
> keys, where each of them is a numpy array.
>
> What I want to achieve is to "trim" these numpy arrays (remove the first
> TRIM
> elements for all the arrays). My attempt goes like this:
>
> for k in self.data_dictionary.iterkeys():
>   for w in self.data_dictionary[k].iterkeys():
>                if w<>'array2d':
>                        self.data_dictionary[k][w] =
> self.data_dictionary[k][w][TRIM:]
>                else:
>                        for b in xrange(7):
>                                self.data_dictionary[k][w][b] =
> self.data_dictionary[k][w][b][TRIM:]
>
>
I haven't put much thought here on approaches, but one thing that should
help is using this for the outer loop:

for k,value in self.data_dictionary.iteritems(): # Gives key, value pairs
from dictionary

And then replace everywhere you use self.data_dictionary[k] with value. That
cuts down on having to look up data_dictionary[k] all over the place.

Ryan

-- 
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/scipy-user/attachments/20090605/79d4aee1/attachment-0001.html 


More information about the SciPy-user mailing list