6.3-exam3-2012¶
Quiz 3, 2012 · Section 6.3: Symmetric Positive Definite Matrices Source: exam3_2.pdf, pp. 33-35
Problem¶
1. (16 pts.)
a) (4 pts.) Suppose \(C\) is \(n \times n\) and positive definite. If \(A\) is \(n \times m\) and \(M = A^TCA\) is not positive definite, find the smallest eigenvalue of \(M\). (Explain briefly.)
b) (12 pts.) If \(A\) is symmetric, which of these four matrices are necessarily positive definite? \(A^3\), \((A^2 + I)^{-1}\), \(A + I\), \(e^A\). (Explain briefly.)
Solution
a) The smallest eigenvalue of \(M\) is 0. First of all, note that \(M^T = A^TC^TA = A^TCA = M\), so \(M\) is symmetric. That implies that all the eigenvalues of \(M\) are real. Since we are assuming that \(M\) is *not* positive definite, at least one of its eigenvalues must be nonpositive. So, to solve the problem, we just have to explain why \(M\) cannot have any negative eigenvalues. The explanation is that \(M\) is positive semidefinite. Why is \(M\) positive semidefinite? Since \(C\) is positive definite, for every vector \(y\) in \(\mathbb{R}^n\) we know that \(y^TCy \geq 0\), with equality if and only if \(y\) is the zero vector. Then, for any vector \(x\) in \(\mathbb{R}^m\), we may set \(y = Ax\), and see that \[ x^TMx = x^TA^TCAx = (Ax)^TC(Ax) \geq 0. \quad (*) \] Since \(M\) is symmetric, the fact that \(x^TMx\) is always nonnegative means that \(M\) is positive semidefinite. Such a matrix never has negative eigenvalues: if \(M\) did have a negative eigenvalue, say \(\lambda < 0\), with a corresponding eigenvector \(v \neq 0\), then \[ v^TMv = v^T(\lambda v) = \lambda\|v\|^2 < 0, \] which would contradict \((*)\), which is supposed to hold for *every* \(x\) in \(\mathbb{R}^m\). (Note: some students wrote that \(M\) is similar to \(C\), but this is totally false. In the given problem, if \(m \neq n\), then \(M\) and \(C\) don't even have the same dimensions, so they cannot possibly be similar.) b) The answer is that \((A^2 + I)^{-1}\) and \(e^A\) have to be positive definite, but \(A^3\) and \(A + I\) don't. The key is to use the \(Q\Lambda Q^{-1}\) factorization. Since \(A\) is symmetric, there is an orthonormal basis of \(\mathbb{R}^n\) consisting of eigenvectors \(q_1, q_2, \dots, q_n\) of \(A\), and the corresponding eigenvalues \(\lambda_1, \lambda_2, \dots, \lambda_n\) are all real. Form an \(n \times n\) matrix \(Q\) whose columns are these eigenvectors, and let \(\Lambda\) be the diagonal matrix with diagonal entries \(\lambda_1, \dots, \lambda_n\), so that \(A = Q\Lambda Q^{-1}\) (equivalently, \(A = Q\Lambda Q^T\), since \(Q\) is orthogonal). All four matrices we are asked to discuss are symmetric, so the question of positive definiteness is just a question about the positivity of their eigenvalues. * \(A^3 = (Q\Lambda Q^{-1})^3 = Q\Lambda^3Q^{-1}\), so \(A^3\) is similar to \(\Lambda^3\), and these two matrices have the same eigenvalues. But \(\Lambda^3\) is the diagonal matrix with diagonal entries \(\lambda_1^3, \dots, \lambda_n^3\), which need not all be positive: for example, \(A = \Lambda = 0\) gives \(A^3 = 0\), which isn't positive definite. * \((A^2 + I)^{-1}\): first note that \(A^2 + I = Q(\Lambda^2 + I)Q^{-1}\), and \(\Lambda^2 + I\) is a diagonal matrix whose diagonal entries \(\lambda_i^2 + 1\) are all nonzero, so it is invertible, and \[ (A^2 + I)^{-1} = Q(\Lambda^2 + I)^{-1}Q^{-1}, \] with eigenvalues \((\lambda_1^2 + 1)^{-1}, \dots, (\lambda_n^2 + 1)^{-1}\), which are all positive because \((\lambda^2 + 1)^{-1} > 0\) for any real number \(\lambda\). So \((A^2 + I)^{-1}\) is positive definite. * \(A + I = Q(\Lambda + I)Q^{-1}\), so \(A + I\) has eigenvalues \(\lambda_1 + 1, \dots, \lambda_n + 1\), which need not all be positive: for example, \(A = -I\) gives \(A + I = 0\), which isn't positive definite. * Finally, \[ e^A = e^{Q\Lambda Q^{-1}} = \sum_{k=0}^{\infty}\frac{1}{k!}(Q\Lambda Q^{-1})^k = Q\left(\sum_{k=0}^{\infty}\frac{1}{k!}\Lambda^k\right)Q^{-1} = Qe^{\Lambda}Q^{-1}, \] so \(e^A\) has eigenvalues \(e^{\lambda_1}, \dots, e^{\lambda_n}\), which are all positive because \(e^{\lambda} > 0\) for all real \(\lambda\). So the eigenvalues of \(e^A\) are all positive, and \(e^A\) must be positive definite. In short: diagonalization reduces the problem to deciding which of the functions \(\lambda^3\), \((\lambda^2 + 1)^{-1}\), \(\lambda + 1\), \(e^\lambda\) is everywhere positive.Keywords: positive definite, positive semidefinite, symmetric matrix, matrix exponential, eigenvalues