[SciPy-user] advice on stochastic(?) optimisation
Thu Aug 28 17:49:10 CDT 2008
On 28-Aug-08, at 11:23 AM, bryan cole wrote:
> I'll looking for a bit of guidance as to what sort of algorithm is
> appropriate/efficient for finding the local maximum of a function
> (in 2
> dimensions), where each function evaluation is 1) noisy and 2)
> expensive/slow to evaluate.
Noisy how, exactly? And do you have gradients (or approximate
gradients)? Can you at least be guaranteed that the function you are
evaluating is proportional (on average) to the true function?
There is a wide and deep literature on stochastic gradient descent,
particularly in the context of neural networks. Here are some papers
that you might find of interest:
Local Gain Adaptation in Stochastic Gradient Descent by N.
A set of lecture notes by Leon Bottou on the subject: http://
In two dimensions, though, I doubt anything too complicated will be
More information about the SciPy-user