4.3-exam2-2008¶
Quiz 2, 2008 · Section 4.3: Least Squares Approximations Source: exam2_1.pdf, pp. 161-162
Problem¶
2 (33 pts.)
(a) Suppose you want to fit the best straight line \(C + Dt\) to the values \(b = 1, 1, 1, 2\) at the times \(t = 0, 1, 3, 4\). What is the matrix \(A\) in the unsolvable system \(A \begin{bmatrix} C \\ D \end{bmatrix} = b\)? Find the best \(\widehat{C}, \widehat{D}\) and the heights \(p_1, p_2, p_3, p_4\) of that line \(\widehat{C} + \widehat{D}t\) at the times \(t = 0, 1, 3, 4\).
(b) Think of the same problem as a projection onto the column space of \(A\) in \(\mathbb{R}^4\). What is the error vector \(e = b - p\)? Show with numbers that \(e\) is perpendicular to (what space?).
(c) Use Gram-Schmidt to get orthonormal columns \(q_1, q_2\) from the columns \(a_1, a_2\) of your matrix \(A\).
Solution
Solution (11+11+11 points) a) The matrix \(A\) is \[ A = \begin{bmatrix} 1 & 0 \\ 1 & 1 \\ 1 & 3 \\ 1 & 4 \end{bmatrix}. \] To find the best \(\widehat{C}, \widehat{D}\), we need to solve the system \[ A^TA \begin{bmatrix} \widehat{C} \\ \widehat{D} \end{bmatrix} = A^T \begin{bmatrix} 1 \\ 1 \\ 1 \\ 2 \end{bmatrix}, \] or \[ \begin{bmatrix} 4 & 8 \\ 8 & 26 \end{bmatrix} \begin{bmatrix} \widehat{C} \\ \widehat{D} \end{bmatrix} = \begin{bmatrix} 5 \\ 12 \end{bmatrix}, \qquad \begin{bmatrix} \widehat{C} \\ \widehat{D} \end{bmatrix} = \frac{1}{40} \begin{bmatrix} 26 & -8 \\ -8 & 4 \end{bmatrix} \begin{bmatrix} 5 \\ 12 \end{bmatrix}. \] We obtain the solution \((\widehat{C}, \widehat{D}) = (17/20, 1/5)\). To find the height vector \(p\), we take \[ p = A \begin{bmatrix} \widehat{C} \\ \widehat{D} \end{bmatrix} = \frac{1}{20}[17, 21, 29, 33]^T. \] b) The error vector is \[ e = [1, 1, 1, 2]^T - p = \frac{1}{20}[3, -1, -9, 7]^T. \] The error vector is perpendicular to the column space of \(A\). We check using numbers: \[ [1, 1, 1, 1]e = \frac{1}{20}(3 - 1 - 9 + 7) = 0, \qquad [0, 1, 3, 4]e = \frac{1}{20}(0 - 1 - 27 + 28) = 0. \] c) We use Gram-Schmidt on the columns of \(A\). We set \(w_1 = a_1\), and then \[ w_2 = a_2 - \frac{w_1 \cdot a_2}{w_1 \cdot w_1}w_1 = [-2, -1, 1, 2]^T. \] Finally, we must normalize to obtain \(q_1 = \frac{1}{2}[1, 1, 1, 1]^T\) and \(q_2 = \frac{1}{\sqrt{10}}[-2, -1, 1, 2]^T\).Keywords: least squares, line fitting, projection, Gram-Schmidt