6.3-exam3-2012-strang¶
Quiz 3, 2012 (strang) · Section 6.3: Symmetric Positive Definite Matrices Source: exam3_2.pdf, pp. 49-52
Problem¶
3. (37 pts.)
(a) Suppose \(A\) is an \(n \times n\) symmetric matrix with eigenvalues \(\lambda_1 \le \lambda_2 \le \dots \le \lambda_n\). What is the largest real number \(c\) that can be subtracted from the diagonal entries of \(A\), so that \(A - cI\) is positive semidefinite? Why?
(b) Suppose \(B\) is a matrix with independent columns. What is the nullspace \(N(B)\)? Show that \(A = B^TB\) is positive definite. Start by saying what that means about \(x^TAx\).
(c) This matrix \(A\) has rank \(r = 1\):
Find its largest singular value \(\sigma\) from \(A^TA\). From its column space and row space, respectively, find unit vectors \(u\) and \(v\) so that
From the nullspaces of \(A\) and \(A^T\) put numbers into the full SVD (Singular Value Decomposition) of \(A\):
Solution
(a) We first realize that if \(A\) is symmetric, then \(A - cI\) is also symmetric, since in general \((A + B)^T = A^T + B^T\). Then we realize that the eigenvalues of \(A - cI\) are \(\lambda_1 - c \le \lambda_2 - c \le \dots \le \lambda_n - c\). Therefore \(c = \lambda_1\) is the largest that can ensure positive semidefiniteness (and it does). (b) Since \(B\) has independent columns, \(Bx = 0\) only has the zero solution, so \(N(B) = \{0\}\). We start by observing that \(A^T = A\) is symmetric. Then we recall what positive definite means (the "energy" test): \[ x^TAx > 0 \quad \text{whenever} \quad x \neq 0. \] By the inner product property of the transpose of a matrix, \[ x^TAx = x^T(B^T(Bx)) = (Bx)^T(Bx) = \|Bx\|^2 \ge 0, \] so \(A = B^TB\) is positive semidefinite. But finally, the equality \(\|Bx\|^2 = 0\) only happens when \(Bx = 0\), which by \(N(B) = \{0\}\) means \(x = 0\). Hence \(A\) is positive definite. (c) We compute: \[ A^TA = \begin{bmatrix} 5 & 5 \\ 5 & 5 \end{bmatrix}. \] Thus the two eigenvalues are \(\lambda_1 = 0\) and \(\lambda_2 = 10\), and \(\sigma = \sqrt{10}\). For \(v\), we find a vector in \(N(A^TA - 10I)\), and normalize to unit length: \[ v = \begin{bmatrix} 1/\sqrt{2} \\ 1/\sqrt{2} \end{bmatrix}. \] Then we find \(u\) using \[ u = \frac{Av}{\sigma} = \begin{bmatrix} 1/\sqrt{5} \\ 2/\sqrt{5} \end{bmatrix}. \] Since we have the orthogonal sums of subspaces \(\mathbb{R}^m = C(A) \oplus N(A^T)\) and also \(\mathbb{R}^n = C(A^T) \oplus N(A)\), we need to find one unit vector from each of \(N(A)\) and \(N(A^T)\) and augment to \(v\) and \(u\), respectively: \[ v_2 = \begin{bmatrix} 1/\sqrt{2} \\ -1/\sqrt{2} \end{bmatrix} \in N(A), \qquad u_2 = \begin{bmatrix} -2/\sqrt{5} \\ 1/\sqrt{5} \end{bmatrix} \in N(A^T). \] Thus, we finally see the full SVD: \[ A = U\Sigma V^T = \begin{bmatrix} 1/\sqrt{5} & -2/\sqrt{5} \\ 2/\sqrt{5} & 1/\sqrt{5} \end{bmatrix}\begin{bmatrix} \sqrt{10} & 0 \\ 0 & 0 \end{bmatrix}\begin{bmatrix} 1/\sqrt{2} & 1/\sqrt{2} \\ 1/\sqrt{2} & -1/\sqrt{2} \end{bmatrix}^T. \] We remember, as a final check, to verify that the square matrices \(U\) and \(V\) both contain orthonormal bases of \(\mathbb{R}^2\) as they should: \(UU^T = I_2\) and \(VV^T = I_2\).Keywords: positive semidefinite, positive definite, eigenvalues, singular value decomposition, nullspace