# [SciPy-User] Optimize fmin_cg issue

Aronne Merrelli aronne.merrelli@gmail....
Sun Jul 29 20:01:25 CDT 2012

On Sun, Jul 29, 2012 at 7:04 PM, Oleksandr Huziy <guziy.sasha@gmail.com> wrote:
> Hi,
>
> I have no idea what is the functio you are minimizing but on my machine I
> get the following:
>
> In [11]: res = optimize.fmin_cg(f, z0, fprime,gtol =1e-5, norm=sp.Inf)
> Optimization terminated successfully.
>          Current function value: 0.000000
>          Iterations: 51
>          Function evaluations: 169
>
> In [12]: res
> Out[12]: array([ 1.00000022,  1.00000044])
>

I get the same result, but I get a very similar NaN result as the OP
if I try with a point farther from zero (try [100,150] instead of
[10,15]). That suggests this is a 32bit vs 64bit float overflow issue
- perhaps OP is using 32 bit and we are using 64 bit floats. So, I
doubt the algorithm has a bug per se, but the output message could be
improved.

I'm not an expert at this part of SciPy, but my guess is that the
iteration loop just isn't checking for NaN values to determine the
"success" of the iterative loop. Looking inside fmin_cg in optimize.py
(moved to _minimize_cg() in the current github), I think what might be
happening is that the gnorm variable is NaN, and then this causes the
while loop to terminate since the while loop's conditional is (gnorm >
gtol), and that will be false. So the later code just sees an exit of
the while loop well short of the iteration maximum, and it only checks
the iteration limit and step size (alpha_k) to determine "success".

[1] https://github.com/scipy/scipy/blob/master/scipy/optimize/optimize.py

Cheers,
Aronne

>
>
> 2012/7/29 The Helmbolds <helmrp@yahoo.com>
>>
>> There appears to be something NQR (not quite right) with the SciPy
>> optimize fmin_cg routine's handling of certain input values. Consider the
>> following. Yes, I know the inputs are odd and far from the true optimal
>> values, but I was experimenting to see what would happen with steep-sloped
>> objective function. In the following, I use a scale factor to multiply the
>> value of the objective function.
>>
>> As you will see, the output flags an error in "double_scalars". I dunno
>> what they are, but assume it's a C-code item. Also, <Items in angle brackets
>> like this are my annotations.>
>>
>>   Enter guessed initial value of x, x0 = 10
>>   Enter guessed initial value of y, y0 = 15
>>   Use scale value, scale = 10
>>                         <End of user inputs . Run description next.>
>>    With f(z) = 10 * optimize.rosen(z) and fprime(z) = 10 *
>> optimize.rosen_der(z),
>>       use SciPy's 'fmin_cg' Polak-Ribiere method to seek the minimum value
>> of f:
>>    Use start value, z0 = [10 15]:
>>       At start, f(z0) = 7225810.0
>>       At start, fprime(z0) = [3400180 -170000])
>>       At start, Inf-norm of fprime(z0) = 3400180
>>    Use the call:
>>       res = sp.optimize.fmin_cg(f, z0, fprime, gtol=1e-05, norm=sp.Inf)
>>                       <End of run description. Output results next.>
>> Warning (from warnings module):
>>   File "C:\Python27\lib\site-packages\scipy\optimize\linesearch.py", line
>> 432
>>     B = (fb-D-C*db)/(db*db)
>> RuntimeWarning: overflow encountered in double_scalars
>> Warning (from warnings module):
>>   File "C:\Python27\lib\site-packages\scipy\optimize\linesearch.py", line
>> 412
>>     A /= denom
>> RuntimeWarning: divide by zero encountered in double_scalars
>> Warning (from warnings module):
>>   File "C:\Python27\lib\site-packages\scipy\optimize\linesearch.py", line
>> 413
>>     B /= denom
>> RuntimeWarning: divide by zero encountered in double_scalars
>> Warning (from warnings module):
>>   File "C:\Python27\lib\site-packages\scipy\optimize\linesearch.py", line
>> 414
>> RuntimeWarning: invalid value encountered in double_scalars
>> Optimization terminated successfully.
>> <Eh!? What!? >
>>          Current function value: nan
>>          Iterations: 7
>>          Function evaluations: 109
>>    fmin_cg returned res = [ nan  nan]
>>    At end, zopt = [ nan  nan]
>>    At end, f(zopt) = nan
>>    At end, fprime(zopt) = [ nan  nan]
>>    At end, Inf-norm of fprime(zopt) = nan
>>
>> _______________________________________________
>> SciPy-User mailing list
>> SciPy-User@scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>
>
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>