[Numpy-discussion] memoization with ndarray arguments

Gregor Thalhammer gregor.thalhammer@gmail....
Sun Mar 22 05:12:39 CDT 2009


Paul Northug schrieb:
> I would like to 'memoize' the objective, derivative and hessian
> functions, each taking a 1d double ndarray argument X, that are passed
> as arguments to
> scipy.optimize.fmin_ncg.
>
> Each of these 3 functions has calculations in common that are
> expensive to compute and are a function of X. It seems fmin_ncg
> computes these quantities at the same X over the course of the
> optimization.
>
> How should I go about doing this?
>   
Exactly for this purpose I was using something like:
cache[tuple(X)] = (subexpression1, subexpression2)
This worked fine for me. In your use case it might be enought to store 
only the latest result to avoid excessive memory  usage, since typically 
the same X is used for consecutive calls of objective, derivative and 
hessian functions.

Gregor
> numpy arrays are not hashable, maybe for a good reason. I tried anyway
> by  keeping a dict of hash(tuple(X)), but started having collisions.
> So I switched to md5.new(X).digest() as the hash function and it seems
> to work ok. In a quick search, I saw cPickle.dumps and repr are also
> used as key values.
>
> I am assuming this is a common problem with functions with numpy array
> arguments and was wondering what the best approach is (including not
> using memoization).
>   



More information about the Numpy-discussion mailing list