Skip to content

7.1-final-2010-strang

Final Exam, 2010 (strang) · Section 7.1: Singular Values and Singular Vectors Source: finalexam_2.pdf, pp. 32-33

Problem

6 (11 points) This problem is about the matrix

\[ A = \begin{bmatrix} 1 & 2 \\ 2 & 4 \\ 3 & 6 \end{bmatrix}. \]

(a) Find the eigenvalues of \(A^T A\) and also of \(AA^T\). For both matrices find a complete set of orthonormal eigenvectors.

(b) If you apply the Gram-Schmidt process (orthonormalization) to the columns of this matrix \(A\), what is the resulting output?

(c) If \(A\) is any \(m\) by \(n\) matrix with \(m > n\), tell me why \(AA^T\) cannot be positive definite. Is \(A^T A\) always positive definite? (If not, what is the test on \(A\)?)

Solution (a) \[ A^T A = \begin{bmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \end{bmatrix} \begin{bmatrix} 1 & 2 \\ 2 & 4 \\ 3 & 6 \end{bmatrix} = \begin{bmatrix} 14 & 28 \\ 28 & 56 \end{bmatrix} \] has eigenvalues \(\lambda_1 = 70\) and \(\lambda_2 = 0\) with orthonormal eigenvectors \[ x_1 = \frac{1}{\sqrt{5}} \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \qquad x_2 = \frac{1}{\sqrt{5}} \begin{bmatrix} -2 \\ 1 \end{bmatrix}. \] Also \[ AA^T = \begin{bmatrix} 1 & 2 \\ 2 & 4 \\ 3 & 6 \end{bmatrix} \begin{bmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \end{bmatrix} = \begin{bmatrix} 5 & 10 & 15 \\ 10 & 20 & 30 \\ 15 & 30 & 45 \end{bmatrix} \] has eigenvalues \(\lambda_1 = 70, \lambda_2 = 0, \lambda_3 = 0\) with orthonormal eigenvectors \[ x_1 = \frac{1}{\sqrt{14}} \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}, \qquad x_2 = \frac{1}{\sqrt{5}} \begin{bmatrix} -2 \\ 1 \\ 0 \end{bmatrix}, \qquad x_3 = \frac{1}{\sqrt{70}} \begin{bmatrix} 3 \\ 6 \\ -5 \end{bmatrix}. \] (b) Gram-Schmidt will find the unit vector \[ q_1 = \frac{1}{\sqrt{14}} \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}. \] But the construction of \(q_2\) fails because column 2 = 2 (column 1). (c) \(AA^T\) is \(m\) by \(m\) but its rank is not greater than \(n\) (all columns of \(AA^T\) are combinations of columns of \(A\)). Since \(n < m\), \(AA^T\) is singular and cannot be positive definite. \(A^T A\) is positive definite exactly when \(A\) has full column rank \(n\). (Not always true: \(A\) can even be a zero matrix.)

Keywords: A^T A and AA^T, orthonormal eigenvectors, Gram-Schmidt, positive definite