# [SciPy-Dev] On the leastsq/curve_fit method

josef.pktd@gmai... josef.pktd@gmai...
Tue Sep 27 09:20:07 CDT 2011

On Tue, Sep 27, 2011 at 5:17 AM, Gianfranco Durin <g.durin@inrim.it> wrote:

>
> > where func can be both "_general_function" and
> > "_weighted_general_function", is not correct.
> >
> >
> > M = σ 2 I ok unit weights
> >
> > M = W^(-1) your case, W has the estimates of the error covariance
> >
> > M = σ 2 W^(-1) I think this is what curve_fit uses, and what is in
> > (my) textbooks defined as weighted least squares
> >
> > Do we use definition 2 or 3 by default? both are reasonable
> >
> > 3 is what I expected when I looked at curve_fit
> > 2 might be more useful for two stage estimation, but I didn't have
> > time to check the details
>
> Ehmm, no, curve_fit does not use def 3, as the errors would scale with W,
> but they don't. By the way, it does not have the correct units.
>

http://wwwasdoc.web.cern.ch/wwwasdoc/minuit/node31.html

'''
The minimization of [image: $\chi^{2}_{}$] above is sometimes called *weighted
least  squares* in which case the inverse quantities 1/*e*2 are called the
weights. Clearly this is simply a different word for the same thing, but in
practice the use of these words sometimes means that the interpretation of *
e*2 as variances or squared errors is not straightforward. The word weight
often implies that only the relative weights are known (point two is twice
as important as point one'') in which case there is apparently an unknown
overall normalization factor. Unfortunately the parameter errors coming out
of such a fit will be proportional to this factor, and the user must be
aware of this in the formulation of his problem.
'''
(I don't quite understand the last sentence.)

M = σ^2 W^(-1), where σ^2 is estimated by residual sum of squares from
weighted regression.

W only specifies relative errors, the assumption is that the covariance
matrix of the errors is *proportional* to W. The scaling is arbitrary. If
the scale of W changes, then the estimated residual sum of squares from
weighted regression will compensate for it. So, rescaling W has no effect on
the covariance of the parameter estimates.

I checked in Greene: Econometric Analysis, and briefly looked at the SAS
description. It looks like weighted least squares is always with automatic
scaling, W is defined only as relative weights.

All I seem to be able to find is weighted least squares with automatic
scaling (except for maybe some two-step estimators).

>
> Curve_fit calculates
>
> M = W \sigma^2 W^(-1) = \sigma^2
>

If I remember correctly (calculated from the transformed model) it should
be:

the cov of the parameter estimates is s^2 (X'WX)^(-1)
error estimates should be s^2 * W

where W = diag(1/curvefit_sigma**2)   unfortunate terminology for
curve_fit's sigma or intentional ? (as I mentioned in the other thread)

Josef

>
> so it gives exactly the same results of case 1, irrespective the W's. This
> is why the errors do not scale with W.
>
> Gianfranco
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/scipy-dev/attachments/20110927/4b103a5a/attachment.html