Pauli Virtanen pav@iki...
Sat May 9 13:01:03 CDT 2009

Sat, 09 May 2009 10:07:55 -0600, Charles R Harris wrote:
> On Sat, May 9, 2009 at 9:55 AM, Pauli Virtanen <pav@iki.fi> wrote:
>>  Anyway, it might be useful to refactor qrfac and qrsolve out of PFIT;
>>  there may be other applications when it's useful to be able to solve
>>  ||(A + I lambda) x - b||_2 = min! efficiently for multiple different
>>  `lambda` in sequence.
>
> This looks like Levenberg-Marquardt. There is a version already in
> MINPACK.

I'm not sure I understand what you mean by the above.

Yes, there's an implementation of Levenberg-Marquardt in MINPACK, but the
Python code discussed here is exactly a Python port of the MINPACK code,
with some additional features bolted on.

For the LSQ problem, course there's numpy.linalg.lstsq, but the point in
the Minpack version is that you can avoid repeated QR-factorization of
the matrix A if you want to find the result for many values of the
parameter lambda. I don't think there's anything like this exposed in
Scipy; of course the MINPACK codes are there, but are they accessible
from Python?

[clip]
> Not exactly though, the version quoted depends on the column ordering of
> A. That doesn't look right.

Ah, I put lambda in the wrong place in the LSQ problem I wrote there, it
should have been

|| (A; sqrt(lambda) I) x - (b; 0)||_2 = min!

which is equivalent to the Levenberg-Marquardt step computation
(A^T A + I lambda) x = A^T b. This should make more sense...

--
Pauli Virtanen