# [SciPy-user] fmin won't optimize

Yaroslav Bulatov yaroslavvb at gmail.com
Mon Nov 8 03:40:53 CST 2004

Sorry, for the double send, I was confused by the "message awaits
moderator approval" replies.

Yes, if I change starting parameter to be [0.,0.,0.] instead of
[0,0,0] fmin and fmin_powell find the same optimum. I guess fmin
doesn't cast them to floats?

Yaroslav

On Sun, 7 Nov 2004 08:30:58 -0700, John Gleeson <jdgleeson at mac.com> wrote:
>
>
>
> On Nov 6, 2004, at 9:58 PM, Travis Oliphant wrote:
>
> > Yaroslav Bulatov wrote:
> >
> >> Hi
> >>
> >> I'm fitting Gibbs distribution to data using maximum likelihood. If I
> >> use fmin_powell, it works, but when I use fmin, it terminates after 1
> >> iteration regardless of starting point. The log-likelihood function is
> >> convex. Any idea why this would happen?
> >>
> >> The code below terminates with likelihood 4.158883 if I use fmin, but
> >> with 3.296169 if "fmin" is replaced with "fmin_powell"
> >>
> >> # Train simple 3 parameter Gibbs Distribution
> >>
> >> import math
> >> from scipy import *
> >> from scipy.optimize import *
> >>
> >> train_set=[(0,0),(0,1),(1,1)]
> >> #test_set=[(0,0),(0,1),(1,1)]
> >>
> >> # Negative log-likelihood
> >> def neg_ll(lambdas):
> >>  unnormalized = lambda(x):
> >>
> > Note, this line break needed to be fixed for it to work.
> >
> >>  math.exp(lambdas[0]*x[0]+lambdas[1]*x[1]+lambdas[2]*x[0]*x[1])
> >>  Z = sum([unnormalized(x) for x in [(0,0),(0,1),(1,0),(1,1)]])
> >>  ll = 0
> >>  for x in train_set:
> >>    ll+=math.log(unnormalized(x))-math.log(Z)
> >>  return -ll
> >>
> >> def main():
> >>  x0 = [0,0,0]
> >>  xopt = fmin(neg_ll, x0)
> >>  print 'Before training: %f' %(neg_ll(x0),)
> >>  print 'After training: %f' %(neg_ll(xopt),)
> >>
> >> if __name__=='__main__': main()
> >>
> >>
> > I tried your code and it worked like this for me:
> >
> > >>> xopt = fmin_powell(neg_ll,x0)
> > Optimization terminated successfully.
> >         Current function value: 3.296169
> >         Iterations: 7
> >         Function evaluations: 168
> > >>> print xopt
> > [-8.2385 -0.0122  8.254 ]
> >
> > >>> xopt = fmin(neg_ll,x0)
> > Optimization terminated successfully.
> >         Current function value: 3.295837
> >         Iterations: 269
> >         Function evaluations: 510
> > >>> print xopt
> > [-34.2701  -0.      34.2701]
> >
>
> Yaroslav, maybe you missed Pearu's response to your
> earlier email?
>
> I get 1 iteration and 4.158883 from fmin also.  But if
> I change x0 = [0,0,0] to x0 = [0.0,0.0,0.0], as Pearu suggested , then
> I get exactly the same
> results as Travis.
>
> John
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.net
> http://www.scipy.net/mailman/listinfo/scipy-user
>