4.4-final-2013-strang¶
Final Exam, 2013 (strang) · Section 4.4: Orthonormal Bases and Gram-Schmidt Source: finalexam_2.pdf, p. 112
Problem¶
3 (5+5 pts.) Suppose \(q_1, \dots, q_5\) are orthonormal vectors in \(\mathbb{R}^5\).
The 5 by 3 matrix \(A\) has columns \(q_1, q_2, q_3\).
(a) If \(b = q_1 + 2q_2 + 3q_3 + 4q_4 + 5q_5\), find the best least squares solution \(\widehat{x}\) to \(Ax = b\).
(b) In terms of \(q_1, q_2, q_3\) find the projection matrix \(P\) onto the column space of \(A\).
Solution
(a) The projection of \(b\) onto the column space \(C(A) = \mathrm{span}\{q_1, q_2, q_3\}\) keeps the components along \(q_1, q_2, q_3\) and drops the components along \(q_4, q_5\), since those are orthogonal to \(C(A)\): \[ \mathrm{proj}_{C(A)} b = q_1 + 2q_2 + 3q_3. \] Since \(A \widehat{x} = \mathrm{proj}_{C(A)} b\) and \(A \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix} = q_1 + 2q_2 + 3q_3\), the best least squares solution is \[ \widehat{x} = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}. \] (b) For a matrix with orthonormal columns, the projection matrix onto its column space is \[ P = A(A^T A)^{-1} A^T = AA^T \quad (\text{since } A^T A = I_3), \] i.e. \[ P = q_1 q_1^T + q_2 q_2^T + q_3 q_3^T. \]Keywords: orthonormal columns, least squares, projection matrix, normal equations