[SciPy-user] Python - Matlab least squares difference

A. Rachel Grinberg agree74 at hotmail.com
Sat Jun 17 16:50:18 CDT 2006

Robert,

You are absolutely right. If the system is underdetermined there are
infinitely many solutions. Still, Matlab's solution yields residual = 0, and
scipy gives you a VERY good approximation, though there are infinitely many
"true"  solutions. I didn't get a chance to actually test out Matlab's
accuracy. Meaning I don't really know that the example I had was a
coincidence, of if Matlab always gives you solution with residual = 0 in
case of an undwerdetermined system. If later is the case, than I would say
that Matlab's algorithm is better.

Rachel

A. Rachel Grinberg wrote:
>Robert,
>
>
>The solution to the least square problem minimizes the l2 norm of the
>residuals.

Just to be clear, my point was that when the least squares problem is
underdetermined, there are an infinite number of vectors that minimize the
L2-norm of the residual equally. It is conventional to regularize the
problem by
choosing the single vector that has the minimum L2-norm from that infinite
set.
Sometimes because it's the choice that makes the most sense, but mostly
because
the SVD gives it to you automatically.

>In my example the residual of Matlab's solution is ||A*(0,0,1)'-b|| = ||0||
>= 0, whereas python's solution yields a number that is very close to zero.

Only because 0.0 and 1.0 and 2.0 can be exactly described in floating point
arithmetic (and presumably only because Matlab pulled its (unconventional)
answer out of thin air magically rather than actually doing the floating
point
calculations that it should be doing). numpy's answer is as close to 0 as
you
can reasonably expect given floating-point arithmetic. Matlab's answer is
not
actually better than numpy's.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it