[Numpy-discussion] matrix default to column vector?

Robert Kern robert.kern@gmail....
Mon Jun 8 15:33:08 CDT 2009


On Mon, Jun 8, 2009 at 15:21, <josef.pktd@gmail.com> wrote:
> 2009/6/8 Stéfan van der Walt <stefan@sun.ac.za>:
>> 2009/6/8 Robert Kern <robert.kern@gmail.com>:
>>>> Remember, the example is a **teaching** example.
>>>
>>> I know. Honestly, I would prefer that teachers skip over the normal
>>> equations entirely and move directly to decomposition approaches. If
>>> you are going to make them implement least-squares from more basic
>>> tools, I think it's more enlightening as a student to start with the
>>> SVD than the normal equations.
>>
>> I agree, and I wish our cirriculum followed that route.  In linear
>> algebra, I also don't much like the way eigenvalues are taught, where
>> students have to solve characteristic polynomials by hand.  When I
>> teach the subject again, I'll pay more attention to these books:
>>
>> Numerical linear algebra by Lloyd Trefethen
>> http://books.google.co.za/books?id=bj-Lu6zjWbEC
>>
>> (e.g. has SVD in Lecture 4)
>>
>> Applied Numerical Linear Algebra by James Demmel
>> http://books.google.co.za/books?id=lr8cFi-YWnIC
>>
>> (e.g. has perturbation theory on page 4)
>>
>> Regards
>> Stéfan
>
> Ok, I also have to give my 2 cents
>
> Any basic econometrics textbook warns of multicollinearity. Since,
> economists are mostly interested in the parameter estimates, the
> covariance matrix needs to have little multicollinearity, otherwise
> the standard errors of the parameters will be huge.
>
> If I use automatically pinv or lstsq, then, unless I look at the
> condition number and singularities, I get estimates that look pretty
> nice, even they have an "arbitrary" choice of the indeterminacy.
>
> So in economics, I never worried too much about the numerical
> precision of the inverse, because, if the correlation matrix is close
> to singular, the model is misspecified, or needs reparameterization or
> the data is useless for the question.
>
> Compared to endogeneity bias for example, or homoscedasticy
> assumptions and so on, the numerical problem is pretty small.
>
> This doesn't mean matrix decomposition methods are not useful for
> numerical calculations and efficiency, but I don't think the numerical
> problem deserves a lot of emphasis in a basic econometrics class.

Actually, my point is a bit broader. Numerics aside, if you are going
to bother peeking under the hood of least-squares at all, I think the
student gets a better understanding of least-squares via one of the
decomposition methods rather than the normal equations.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


More information about the Numpy-discussion mailing list