[SciPy-user] Python - Matlab least squares difference

A. Rachel Grinberg agree74 at hotmail.com
Sat Jun 17 16:50:18 CDT 2006


You are absolutely right. If the system is underdetermined there are 
infinitely many solutions. Still, Matlab's solution yields residual = 0, and 
scipy gives you a VERY good approximation, though there are infinitely many 
"true"  solutions. I didn't get a chance to actually test out Matlab's 
accuracy. Meaning I don't really know that the example I had was a 
coincidence, of if Matlab always gives you solution with residual = 0 in 
case of an undwerdetermined system. If later is the case, than I would say 
that Matlab's algorithm is better.


A. Rachel Grinberg wrote:
>Thanks for your response.
>The solution to the least square problem minimizes the l2 norm of the 

Just to be clear, my point was that when the least squares problem is
underdetermined, there are an infinite number of vectors that minimize the
L2-norm of the residual equally. It is conventional to regularize the 
problem by
choosing the single vector that has the minimum L2-norm from that infinite 
Sometimes because it's the choice that makes the most sense, but mostly 
the SVD gives it to you automatically.

>In my example the residual of Matlab's solution is ||A*(0,0,1)'-b|| = ||0|| 
>= 0, whereas python's solution yields a number that is very close to zero.

Only because 0.0 and 1.0 and 2.0 can be exactly described in floating point
arithmetic (and presumably only because Matlab pulled its (unconventional)
answer out of thin air magically rather than actually doing the floating 
calculations that it should be doing). numpy's answer is as close to 0 as 
can reasonably expect given floating-point arithmetic. Matlab's answer is 
actually better than numpy's.

Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it 
an underlying truth."
  -- Umberto Eco

Don’t just search. Find. Check out the new MSN Search! 

More information about the SciPy-user mailing list