[Numpy-discussion] inversion of large matrices

Daniel Elliott danelliottster@gmail....
Wed Sep 8 15:32:56 CDT 2010

Wow, this is great!  Thanks for all the great questions.

Sebastian Walter <sebastian.walter <at> gmail.com> writes:

> is it really the covariance matrix you want to invert? Or do you want
> to compute something like
> x^T C^{-1} x,
> where x is an array of size N and C an array of size (N,N)?

Yes, this is what I am computing.  I am computing the pdf of a very high-
dimensional multivariate normal.  Is there a specialized method to compute this?
> It would also be interesting to know how the covariance matrix gets computed
> and what its condition number is, at least approximately.

Chuck and Sebastian are on to me.  I should look into computing this using the 
small covariance matrix.  I suppose I could even postpone evaluation of this 
covariance matrix by passing in the data matrix instead of the covariance 
matrix.  What do you guys think?

As for the condition number, the matrix will, initially, be ill-conditioned 
because the dimension of the data will far outnumber the number of data points.  
However, I am writing this code on the way to writing code that fits a mixture 
of Gaussians but can use any number of covariance regularization techniques to 
"fix" the covariance matrices.

Basically, I am writing code that does this: 
http://www.cs.colostate.edu/~dane/papers/ANNIE2010.pdf.  I am guessing you math 
and stats guys might shake your head at my clumsy, computer science ways... :)

However, in addition to the two, simple regularization techniques described in 
that paper, I also want to be able to do things like mixture of factor analyzers 
and mixture of principal component analyzers and so on by simply supplying 
different regularization techniques.

FYI, I am currently adding this to PyMix.  Does this seem like a good package to 

> On Wed, Sep 1, 2010 at 1:58 AM, Charles R Harris
> <charlesr.harris <at> gmail.com> wrote:
> > On Tue, Aug 31, 2010 at 4:52 PM, Dan Elliott <danelliottster <at> gmail.com>
> > wrote:
> >> David Warde-Farley <dwf <at> cs.toronto.edu> writes:
> >> > On 2010-08-30, at 10:36 PM, Charles R Harris wrote:
> >> > I think he means that if he needs both the determinant and to solve the
> >> > system, it might be more efficient to do
> >> > the SVD, obtain the determinant from the diagonal values, and obtain the
> >> > solution by multiplying by U D^-1 V^T?
> >> Thank you, that is what I meant.  Poorly worded on my part.
> >>
> >> In particular, I am writing code to invert a very large covariance matrix.
> >>  I
> >> think David has some good information in another post in this thread.
> >>
> >
> > Where did the covariance array come from? It may be the case that you can
> > use a much smaller one, for instance in PCA of images.
> >
> > Chuck

More information about the NumPy-Discussion mailing list