[SciPy-user] Python - Matlab least squares difference

Robert Kern robert.kern at gmail.com
Sat Jun 17 02:07:57 CDT 2006

A. Rachel Grinberg wrote:
> Robert,
> Thanks for your response.
> The solution to the least square problem minimizes the l2 norm of the 
> residuals.

Just to be clear, my point was that when the least squares problem is
underdetermined, there are an infinite number of vectors that minimize the
L2-norm of the residual equally. It is conventional to regularize the problem by
choosing the single vector that has the minimum L2-norm from that infinite set.
Sometimes because it's the choice that makes the most sense, but mostly because
the SVD gives it to you automatically.

> In my example the residual of Matlab's solution is 
> ||A*(0,0,1)'-b|| = ||0|| = 0, whereas python's solution yields a number that 
> is very close to zero.

Only because 0.0 and 1.0 and 2.0 can be exactly described in floating point
arithmetic (and presumably only because Matlab pulled its (unconventional)
answer out of thin air magically rather than actually doing the floating point
calculations that it should be doing). numpy's answer is as close to 0 as you
can reasonably expect given floating-point arithmetic. Matlab's answer is not
actually better than numpy's.

Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

More information about the SciPy-user mailing list