4.3-final-2008-strang¶
Final Exam, 2008 (strang) · Section 4.3: Least Squares Approximations Source: finalexam_1.pdf, p. 221
Problem¶
10 (11 pts.) This problem uses least squares to find the plane \(C + Dx + Ey = b\) that best fits these 4 points:
\(x = 0\) \(y = 0\) \(b = 2\)
\(x = 1\) \(y = 1\) \(b = 1\)
\(x = 1\) \(y = -1\) \(b = 0\)
\(x = -2\) \(y = 0\) \(b = 1\)
(a) Write down 4 equations \(Ax = b\) with unknown \(x = (C, D, E)\) that would hold if the plane went through the 4 points. Then write down the equations to solve for the best (least squares) solution \(\widehat{x} = (\widehat{C}, \widehat{D}, \widehat{E})\).
(b) Find the best \(\widehat{x}\) and the error vector \(e\) (is the vector e in \(R^3\) or \(R^4\)?).
(c) If you change this \(b = (2, 1, 0, 1)\) to the vector \(p = A\widehat{x}\), what will be the best plane to fit these four new points \((p_1, p_2, p_3, p_4)\)? What will be the new error vector?
Solution
(a) The equations are of the form \(C + 0D + 0E = 2\), etc., or in matrix form \[ \begin{bmatrix} 1 & 0 & 0 \\ 1 & 1 & 1 \\ 1 & 1 & -1 \\ 1 & -2 & 0 \end{bmatrix} \begin{bmatrix} C \\ D \\ E \end{bmatrix} = \begin{bmatrix} 2 \\ 1 \\ 0 \\ 1 \end{bmatrix}. \] Of course this system is not solvable. The best solution is given by the normal equations \(A^TA\widehat{x} = A^Tb\). (b) We have \[ A^TA = \begin{bmatrix} 4 & 0 & 0 \\ 0 & 6 & 0 \\ 0 & 0 & 2 \end{bmatrix} \quad \text{and} \quad A^Tb = \begin{bmatrix} 4 \\ -1 \\ 1 \end{bmatrix}. \] It is a diagonal system, so we immediately find \((C, D, E) = (1, -1/6, 1/2)\). The error vector is the difference of the real \(b\) and the approximate values we get for our plane: \(e = b - A\widehat{x}\). Since \(A\widehat{x} = [1, 4/3, 1/3, 4/3]^T\), we get \(e = (1, -1/3, -1/3, -1/3)\). The vector \(e\) is in \(R^4\). (c) We know \(p = A\widehat{x}\) is the projection of \(b\) onto the column space of \(A\). So the system \(Ax = p\) is solvable exactly; we don't need any approximations. The best fit plane will be the same plane as in part (b): \(1 - x/6 + y/2 = b\) (we changed the b-coordinates of the points so that they lie on this plane, so of course it is the best fit). The error vector will become 0 because it is an exact fit.Keywords: least squares, best fit plane, normal equations, error vector