4.2-exam2-2009-postnikov¶
Quiz 2, 2009 (postnikov) · Section 4.2: Projections onto Lines and Subspaces Source: exam2_1.pdf, pp. 185-186
Problem¶
Problem 2. Let \(A = \begin{bmatrix} 2 & -1 \\ 1 & 2 \\ 0 & -1 \end{bmatrix}\), and let \(b = \begin{bmatrix} 0 \\ 0 \\ 6 \end{bmatrix}\).
(a) What is the projection of \(b\) onto the column space of \(A\)?
(b) Give an orthogonal basis for each of the four fundamental subspaces of \(A\).
(c) Use least squares approximation to solve \(Ax = b\).
Solution
(a) Want to find \(p = A\hat{x}\) (i.e. in \(C(A)\)) closest to \(b\). Or equivalently \(A^T(b - A\hat{x}) = 0 \implies A^TA\hat{x} = A^Tb\). Since the columns of \(A\) are independent, \(\hat{x} = (A^TA)^{-1}A^Tb\), so the projection \(A\hat{x}\) is given by \[ p = A(A^TA)^{-1}A^Tb. \] \[ A^TA = \begin{bmatrix} 2 & 1 & 0 \\ -1 & 2 & -1 \end{bmatrix} \begin{bmatrix} 2 & -1 \\ 1 & 2 \\ 0 & -1 \end{bmatrix} = \begin{bmatrix} 5 & 0 \\ 0 & 6 \end{bmatrix}, \] so \((A^TA)^{-1} = \begin{bmatrix} 1/5 & 0 \\ 0 & 1/6 \end{bmatrix}\). Also \[ A^Tb = \begin{bmatrix} 2 & 1 & 0 \\ -1 & 2 & -1 \end{bmatrix} \begin{bmatrix} 0 \\ 0 \\ 6 \end{bmatrix} = \begin{bmatrix} 0 \\ -6 \end{bmatrix}, \qquad \hat{x} = \begin{bmatrix} 1/5 & 0 \\ 0 & 1/6 \end{bmatrix} \begin{bmatrix} 0 \\ -6 \end{bmatrix} = \begin{bmatrix} 0 \\ -1 \end{bmatrix}, \] and therefore \[ p = \begin{bmatrix} 2 & -1 \\ 1 & 2 \\ 0 & -1 \end{bmatrix} \begin{bmatrix} 0 \\ -1 \end{bmatrix} = \begin{bmatrix} 1 \\ -2 \\ 1 \end{bmatrix}. \] (b) From part (a), \(A^TA = \begin{bmatrix} 5 & 0 \\ 0 & 6 \end{bmatrix}\) shows that the columns of \(A\) are already orthogonal. They are clearly independent as \(A\) has 2 pivots, so \(C(A)\) has the (orthogonal) basis \[ \left\{ \begin{bmatrix} 2 \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} -1 \\ 2 \\ -1 \end{bmatrix} \right\}. \] Since \(\dim C(A) = \dim C(A^T) = 2\), the row space is all of \(\mathbb{R}^2\), so we can take any basis; \(C(A^T)\) has the orthogonal basis \(\left\{ \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 1 \end{bmatrix} \right\}\). \(N(A) = \{0\}\) since \(A\) has full column rank. \(N(A^T)\) is spanned by a vector \(v\) such that \(A^Tv = 0\), i.e. \(2v_1 + v_2 = 0\) and \(-v_1 + 2v_2 - v_3 = 0\); \(\left\{ \begin{bmatrix} 1 \\ -2 \\ -5 \end{bmatrix} \right\}\) works. (c) Want to solve \(A^TA\hat{x} = A^Tb\), or \(\hat{x} = (A^TA)^{-1}A^Tb\). From part (a), \[ \hat{x} = \begin{bmatrix} 1/5 & 0 \\ 0 & 1/6 \end{bmatrix} \begin{bmatrix} 0 \\ -6 \end{bmatrix} = \begin{bmatrix} 0 \\ -1 \end{bmatrix}. \]Keywords: projection, four subspaces, least squares, normal equations