[SciPy-dev] ticket 390 ("scipy.optimize.fmin_bfgs fails without warning", Reported by: ycopin)

[dmitrey]

Thank you, however, there are no needs to read anything for to 
understand that classic bfgs is for local minimum only. Even 2 first 
lines of your url already says
*
Broyden–Fletcher–Goldfarb–Shanno method,*
/BFGS method/The unconstrained optimization problem is to minimize a 
real-valued function of variables. That is, to find a *local* minimizer, 
i.e. a point such that ...

We can talk about global convergence of some bfgs modifications ONLY if 
some assumptions (usually very strong) about objfunc are known, for 
example f is Lipschitz twice continuously differentiable.

BFGS uses gradient and is intended for medium and rather large scale 
objfunc, it's already enough for to understand that it's for local 
minimum only (w/o any other assumptions about f, any global solver is 
capable only for small-scale funcs, nVars up to 10-12).

BTW in ticket 344 you were right about the bug, but on the other hand if 
you intend to find global minimum then you also use incorrect solver(s) 
(fmin_cg and fmin_powell) to your objfunc, that has 2 global 
(x=1.32733353855 and x=-1.32733353855, because your objfunc d(x) is 
zero-symmetric), non-convex and lots of local minima.

def g(x):
return 1./(1-cos(x))
def d(x):
return sqrt(x**2+(g(x)-1.0)**2)

HTH, D.

Nils Wagner wrote:
> Alan G Isaac wrote:
>   
>> On Wed, 18 Jul 2007, dmitrey apparently wrote:
>>   
>>     
>>> http://projects.scipy.org/scipy/scipy/ticket/390
>>> So, since user has sin(const/b), the func is 
>>> 1) non-convex 
>>> 2) has lots of local minimum 
>>> So I guess it's nothing special that sometimes fmin_bfgs yields other 
>>> solution (from other start points) than the one the user mentioned. 
>>> So I think the ticket should be closed. 
>>>     
>>>       
>> Yannick, can you comment please?  Nils too.
>> I read the ticket as concerning not the particular point
>> found but that stopping was away from a local
>> minimum without warning, even though the Jacobian was ok.
>>
>> Alan Isaac
>>
>> PS Dmitrey is working on these tickets this week and needs
>> very quick feedback.
>>
>>
>> _______________________________________________
>> Scipy-dev mailing list
>> Scipy-dev@scipy.org
>> http://projects.scipy.org/mailman/listinfo/scipy-dev
>>   
>>     
> Hi Alan,
>
> I am not an expert wrt. optimization.
> The following link might be useful to gain further insight into local
> and global
> convergence properties of BFGS.
> http://eom.springer.de/b/b120510.htm
>
>
> Cheers,
>                       Nils
>  
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev@scipy.org
> http://projects.scipy.org/mailman/listinfo/scipy-dev
>
>
>
>