Skip to content

4.3-final-1999-strang

Final Exam, 1999 (strang) · Section 4.3: Least Squares Approximations Source: finalexam_1.pdf, pp. 34, 40-41

Problem

5. This problem computes the plane \(z = Cx + Dy + E\) that is closest (in the least squares sense) to these four measurements:

At \(x = 1\), \(y = 0\) measurement gives \(z = 1\)

At \(x = 1\), \(y = 2\) measurement gives \(z = 3\)

At \(x = 0\), \(y = 1\) measurement gives \(z = 5\)

At \(x = 0\), \(y = 2\) measurement gives \(z = 0\)

(a) (3 points) Write down the linear system \(Ax = b\) with unknown vector \(x = (C, D, E)\) that would give a plane going exactly through the four given points -- except that this particular system has no solution.

(b) (3 points) Show that this system \(Ax = b\) has no solution!

(c) (3 points) Find the best least squares solution \(\hat{x} = (\widehat{C}, \widehat{D}, \widehat{E})\).

(d) (3 points) The error vector \(e = (e_1, e_2, e_3, e_4)\) in the underlying projection problem is perpendicular to which vectors? You don't have to compute \(e\) but you do have to say which specific numerical vectors it is perpendicular to.

Solution (a) \(1C + 0D + E = 1\) \(1C + 2D + E = 3\) is \(Ax = b\). \(0C + 1D + E = 5\) \(0C + 2D + E = 0\) (b) Subtract equation (1) from equation (2): \(2D = 2\) gives \(D = 1\) \(D + E = 5\) gives \(E = 4\) \(2D + E = 0\) is now false (c) Solve \(A^T A \hat{x} = A^T b\): \[ \begin{bmatrix} 1 & 1 & 0 & 0 \\ 0 & 2 & 1 & 2 \\ 1 & 1 & 1 & 1 \end{bmatrix} \begin{bmatrix} 1 & 0 & 1 \\ 1 & 2 & 1 \\ 0 & 1 & 1 \\ 0 & 2 & 1 \end{bmatrix} \begin{bmatrix} \hat{C} \\ \hat{D} \\ \hat{E} \end{bmatrix} = \begin{bmatrix} 1 & 1 & 0 & 0 \\ 0 & 2 & 1 & 2 \\ 1 & 1 & 1 & 1 \end{bmatrix} \begin{bmatrix} 1 \\ 3 \\ 5 \\ 0 \end{bmatrix} \] \[ \begin{bmatrix} 2 & 2 & 2 \\ 2 & 9 & 5 \\ 2 & 5 & 4 \end{bmatrix} \begin{bmatrix} \hat{C} \\ \hat{D} \\ \hat{E} \end{bmatrix} = \begin{bmatrix} 4 \\ 11 \\ 9 \end{bmatrix} \] \[ \begin{bmatrix} 2 & 2 & 2 \\ 0 & 7 & 3 \\ 0 & 0 & \frac{5}{7} \end{bmatrix} \begin{bmatrix} \hat{C} \\ \hat{D} \\ \hat{E} \end{bmatrix} = \begin{bmatrix} 4 \\ 7 \\ 2 \end{bmatrix} \] Back-substitution gives \(\hat{E} = \frac{14}{5}\), \(\hat{D} = \frac{-1}{5}\), \(\hat{C} = \frac{-3}{5}\). (d) The error vector \(e\) is perpendicular to the three columns of \(A\).

Keywords: least squares, normal equations, projection, error vector