[Scipy-tickets] [SciPy] #1157: Correlation matrix from scipy.optimize.leastsq(): undocumented approximation?

SciPy Trac scipy-tickets@scipy....
Tue Apr 20 16:10:58 CDT 2010


#1157: Correlation matrix from scipy.optimize.leastsq(): undocumented
approximation?
-----------------------------------------+----------------------------------
 Reporter:  lebigot                      |       Owner:  somebody
     Type:  enhancement                  |      Status:  new     
 Priority:  normal                       |   Milestone:  0.8.0   
Component:  scipy.optimize               |     Version:  0.7.0   
 Keywords:  leastsq, correlation matrix  |  
-----------------------------------------+----------------------------------

Comment(by lebigot):

 Here is an example, with arguments to leastsq() that are not of the form
 y-f(x):

 {{{
 from numpy import *
 from scipy.optimize import leastsq

 offset = 10
 leastsq(lambda x: sqrt(offset+(array([2, 0])-x[0])**2), 0,
 full_output=1)[0:2]  # Returns (x, cov)
 }}}

 The minimized function is 2*offset + (2-x)**2 + (0-x)**2.  The minimum is
 obtained for x = 1.  When the offset changes, the variance of the fit
 result changes.

 I did not expect this because I thought that leastsq() was quite general,
 and that the formal covariance matrix that it returned was simply the
 inverse of the Hessian (the matrix of second order derivatives of the
 minimized function).  This matrix does not depend on the offset, in the
 example above.  My expectation comes from Numerical Recipes (for Fortran,
 2nd edition), § 15.6, where the errors on the fitted parameters are
 expressed in terms of the curvature matrix.

 I think that the documentation of the meaning of the covariance matrix in
 leastsq() could usefully be made more precise.  As far as I understand,
 leastsq()'s current implementation uses an approximation that relies on a
 minimized function of the form I was giving (the usual Gaussian log-
 likelihood).  If I remember correctly, there is a discussion in § 14.5 in
 Numerical Recipes (Fortran, 2nd ed.) about approximating the Hessian
 through the Jacobian, which I guess is what is done in leastsq(); if this
 is the case, it would be great if some sufficient conditions on the
 minimized function were added to the documentation of the covariance
 matrix calculated by leastsq(), so that the user knows that he has to
 respect them. :)

-- 
Ticket URL: <http://projects.scipy.org/scipy/ticket/1157#comment:3>
SciPy <http://www.scipy.org>
SciPy is open-source software for mathematics, science, and engineering.


More information about the Scipy-tickets mailing list