4.3-exam2-2006¶
Quiz 2, 2006 · Section 4.3: Least Squares Approximations Source: exam2_1.pdf, pp. 130, 139
Problem¶
2 (24 pts.) Suppose we take measurements at the 21 equally spaced times \(t = -10, -9, \ldots, 9, 10\). All measurements are \(b_i = 0\) except that \(b_{11} = 1\) at the middle time \(t = 0\).
(a) Using least squares, what are the best \(\hat{C}\) and \(\hat{D}\) to fit those 21 points by a straight line \(C + Dt\)?
(b) You are projecting the vector \(b\) onto what subspace? (Give a basis.) Find a nonzero vector perpendicular to that subspace.
Solution
2a. The 21 equations \(Ax = b\) would be \[ \begin{bmatrix} 1 & -10 \\ \vdots & \vdots \\ 1 & 0 \\ \vdots & \vdots \\ 1 & 10 \end{bmatrix} \begin{bmatrix} C \\ D \end{bmatrix} = \begin{bmatrix} 0 \\ \vdots \\ 1 \\ \vdots \\ 0 \end{bmatrix}, \] with the single 1 in the middle row. The least-squares solution solves \(A^T A \hat{x} = A^T b\), and since \(\sum t = 0\) and \(\sum t^2 = 2(1^2 + \cdots + 10^2) = 770\), \[ A^T A = \begin{bmatrix} 21 & 0 \\ 0 & 770 \end{bmatrix}, \qquad A^T b = \begin{bmatrix} 1 \\ 0 \end{bmatrix}. \] Therefore \(\hat{C} = \frac{1}{21}\) and \(\hat{D} = 0\): the line of best fit is the horizontal line \(C = \frac{1}{21}\). 2b. We are projecting \(b\) onto the column space of the matrix \(A\) above, with basis \(\{(1, 1, \ldots, 1)^T, (-10, -9, \ldots, 10)^T\}\). One nonzero vector perpendicular to that subspace is the error vector \[ e = b - P_A b = \frac{1}{21} (\underbrace{-1, \ldots, -1}_{10}, 20, \underbrace{-1, \ldots, -1}_{10})^T, \] which has the 20 in the middle position.Keywords: least squares, straight line fit, projection, error vector