[SciPy-user] fmin won't optimize
John Gleeson
jdgleeson at mac.com
Sun Nov 7 09:30:58 CST 2004
On Nov 6, 2004, at 9:58 PM, Travis Oliphant wrote:
> Yaroslav Bulatov wrote:
>
>> Hi
>>
>> I'm fitting Gibbs distribution to data using maximum likelihood. If I
>> use fmin_powell, it works, but when I use fmin, it terminates after 1
>> iteration regardless of starting point. The log-likelihood function is
>> convex. Any idea why this would happen?
>>
>> The code below terminates with likelihood 4.158883 if I use fmin, but
>> with 3.296169 if "fmin" is replaced with "fmin_powell"
>>
>> # Train simple 3 parameter Gibbs Distribution
>>
>> import math
>> from scipy import *
>> from scipy.optimize import *
>>
>> train_set=[(0,0),(0,1),(1,1)]
>> #test_set=[(0,0),(0,1),(1,1)]
>>
>> # Negative log-likelihood
>> def neg_ll(lambdas):
>> unnormalized = lambda(x):
>>
> Note, this line break needed to be fixed for it to work.
>
>> math.exp(lambdas[0]*x[0]+lambdas[1]*x[1]+lambdas[2]*x[0]*x[1])
>> Z = sum([unnormalized(x) for x in [(0,0),(0,1),(1,0),(1,1)]])
>> ll = 0
>> for x in train_set:
>> ll+=math.log(unnormalized(x))-math.log(Z)
>> return -ll
>>
>> def main():
>> x0 = [0,0,0]
>> xopt = fmin(neg_ll, x0)
>> print 'Before training: %f' %(neg_ll(x0),)
>> print 'After training: %f' %(neg_ll(xopt),)
>>
>> if __name__=='__main__': main()
>>
>>
> I tried your code and it worked like this for me:
>
> >>> xopt = fmin_powell(neg_ll,x0)
> Optimization terminated successfully.
> Current function value: 3.296169
> Iterations: 7
> Function evaluations: 168
> >>> print xopt
> [-8.2385 -0.0122 8.254 ]
>
> >>> xopt = fmin(neg_ll,x0)
> Optimization terminated successfully.
> Current function value: 3.295837
> Iterations: 269
> Function evaluations: 510
> >>> print xopt
> [-34.2701 -0. 34.2701]
>
Yaroslav, maybe you missed Pearu's response to your
earlier email?
I get 1 iteration and 4.158883 from fmin also. But if
I change x0 = [0,0,0] to x0 = [0.0,0.0,0.0], as Pearu suggested , then
I get exactly the same
results as Travis.
John
More information about the SciPy-user
mailing list