Skip to content

4.2-final-2009

Final Exam, 2009 · Section 4.2: Projections onto Lines and Subspaces Source: finalexam_1.pdf, p. 225

Problem

Problem 2: For the matrix \(A = \begin{bmatrix} 1 & 0 & -1 \\ 1 & 1 & 1 \\ 2 & 1 & 0 \end{bmatrix}\) with rank 2, consider the system of equations \(Ax = b\).

(i) \(Ax = b\) has a solution whenever \(b\) is orthogonal to some nonzero vector \(c\). Explicitly compute such a vector \(c\). Your answer can be multiplied by any overall constant, because \(c\) is any basis for the ____ space of \(A\).

(ii) Find the orthogonal projection \(p\) of the vector \(b = \begin{bmatrix} 9 \\ 9 \\ 9 \end{bmatrix}\) onto \(C(A)\). (Note: The matrix \(A^TA\) is singular, so you cannot use your formula \(P = A(A^TA)^{-1}A^T\) to obtain the projection matrix \(P\) onto the column space of \(A\). But I have repeatedly discouraged you from computing \(P\) explicitly, so you don't need to be reminded anyway, right?)

(iii) If \(p\) is your answer from (ii), then a solution \(y\) of \(Ay = p\) minimizes what? [You need not answer (ii) or compute \(y\) for this part.]

Solution (18 points = 7 + 7 + 4) (i) The system of equations \(Ax = b\) has a solution if and only if \(b\) lies in the column space of \(A\), which is orthogonal to the left nullspace of \(A\). We solve for a (nonzero) vector \(c\) in the left nullspace using Gaussian elimination, as follows. \[ A^T = \begin{bmatrix} 1 & 1 & 2 \\ 0 & 1 & 1 \\ -1 & 1 & 0 \end{bmatrix} \rightsquigarrow \begin{bmatrix} 1 & 1 & 2 \\ 0 & 1 & 1 \\ 0 & 2 & 2 \end{bmatrix} \rightsquigarrow \begin{bmatrix} 1 & 0 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{bmatrix} \Rightarrow c = \begin{bmatrix} -1 \\ -1 \\ 1 \end{bmatrix}. \] The answer can be any nonzero multiple of \(c\), which will be a basis for the *left nullspace* of \(A\). (ii) Method 1: Since \(c\) is a basis of the orthogonal complement of the column space \(C(A)\), the projection of \(b\) onto \(C(A)\) can be computed as \[ p = b - \frac{c^Tb}{\|c\|^2}c = \begin{bmatrix} 9 \\ 9 \\ 9 \end{bmatrix} - \frac{-9}{3} \begin{bmatrix} -1 \\ -1 \\ 1 \end{bmatrix} = \begin{bmatrix} 6 \\ 6 \\ 12 \end{bmatrix}. \] Method 2 (not recommended): We know that \(p\) is the best linear approximation of \(b\). So we solve \[ A^TA \begin{bmatrix} y_1 \\ y_2 \\ y_3 \end{bmatrix} = A^T \begin{bmatrix} 9 \\ 9 \\ 9 \end{bmatrix}, \qquad \begin{bmatrix} 6 & 3 & 0 \\ 3 & 2 & 1 \\ 0 & 1 & 2 \end{bmatrix} \begin{bmatrix} y_1 \\ y_2 \\ y_3 \end{bmatrix} = \begin{bmatrix} 36 \\ 18 \\ 0 \end{bmatrix}. \] We can get a particular solution \(y = (6, 0, 0)^T\). (There are other solutions too.) Hence, \[ p = A \begin{bmatrix} y_1 \\ y_2 \\ y_3 \end{bmatrix} = \begin{bmatrix} 1 & 0 & -1 \\ 1 & 1 & 1 \\ 2 & 1 & 0 \end{bmatrix} \begin{bmatrix} 6 \\ 0 \\ 0 \end{bmatrix} = \begin{bmatrix} 6 \\ 6 \\ 12 \end{bmatrix}. \] (iii) Since \(p\) is the orthogonal projection of \(b\) onto \(C(A)\), a solution \(y\) of \(Ay = p\) minimizes the distance \(\|Ay - b\|\).

Keywords: left nullspace, projection, column space, least squares