[Scipy-tickets] [SciPy] #1157: Correlation matrix from scipy.optimize.leastsq(): undocumented approximation?
SciPy Trac
scipy-tickets@scipy....
Tue Apr 20 16:10:58 CDT 2010
#1157: Correlation matrix from scipy.optimize.leastsq(): undocumented
approximation?
-----------------------------------------+----------------------------------
Reporter: lebigot | Owner: somebody
Type: enhancement | Status: new
Priority: normal | Milestone: 0.8.0
Component: scipy.optimize | Version: 0.7.0
Keywords: leastsq, correlation matrix |
-----------------------------------------+----------------------------------
Comment(by lebigot):
Here is an example, with arguments to leastsq() that are not of the form
y-f(x):
{{{
from numpy import *
from scipy.optimize import leastsq
offset = 10
leastsq(lambda x: sqrt(offset+(array([2, 0])-x[0])**2), 0,
full_output=1)[0:2] # Returns (x, cov)
}}}
The minimized function is 2*offset + (2-x)**2 + (0-x)**2. The minimum is
obtained for x = 1. When the offset changes, the variance of the fit
result changes.
I did not expect this because I thought that leastsq() was quite general,
and that the formal covariance matrix that it returned was simply the
inverse of the Hessian (the matrix of second order derivatives of the
minimized function). This matrix does not depend on the offset, in the
example above. My expectation comes from Numerical Recipes (for Fortran,
2nd edition), § 15.6, where the errors on the fitted parameters are
expressed in terms of the curvature matrix.
I think that the documentation of the meaning of the covariance matrix in
leastsq() could usefully be made more precise. As far as I understand,
leastsq()'s current implementation uses an approximation that relies on a
minimized function of the form I was giving (the usual Gaussian log-
likelihood). If I remember correctly, there is a discussion in § 14.5 in
Numerical Recipes (Fortran, 2nd ed.) about approximating the Hessian
through the Jacobian, which I guess is what is done in leastsq(); if this
is the case, it would be great if some sufficient conditions on the
minimized function were added to the documentation of the covariance
matrix calculated by leastsq(), so that the user knows that he has to
respect them. :)
--
Ticket URL: <http://projects.scipy.org/scipy/ticket/1157#comment:3>
SciPy <http://www.scipy.org>
SciPy is open-source software for mathematics, science, and engineering.
More information about the Scipy-tickets
mailing list