Skip to content

4.3-final-1998

Final Exam, 1998 · Section 4.3: Least Squares Approximations Source: finalexam_1.pdf, pp. 8, 12

Problem

7. (a) (4 points) If these three points happen to lie on a line \(y = C + Dt\), what system \(Ax = b\) of three equations in two unknowns would be solvable?

\[ y = 0 \text{ at } t = -1, \quad y = 1 \text{ at } t = 0, \quad y = B \text{ at } t = 1. \]

Which value of \(B\) puts the vector \(b = (0, 1, B)\) into the column space of \(A\)?

(b) (4 points) For every \(B\) find the numbers \(\overline{C}\) and \(\overline{D}\) that give the best straight line \(y = \overline{C} + \overline{D}t\) (closest to the three points in the least squares sense).

(c) (4 points) Find the projection of \(b = (1, 0, 0)\) onto the column space of \(A\).

(d) (2 points) If you apply the Gram-Schmidt procedure to this matrix \(A\), what is the resulting matrix \(Q\) that has orthonormal columns?

Solution (a) The system \(Ax = b\) is \[ \begin{bmatrix} 1 & -1 \\ 1 & 0 \\ 1 & 1 \end{bmatrix} \begin{bmatrix} C \\ D \end{bmatrix} = \begin{bmatrix} 0 \\ 1 \\ B \end{bmatrix}. \] This is solvable if \(B = 2\): the first two equations give \(C = 1\), \(D = 1\), and the third equation \(C + D = 2\) is then satisfied. (b) The normal equations \(A^TA\bar{x} = A^Tb\) are \[ \begin{bmatrix} 3 & 0 \\ 0 & 2 \end{bmatrix} \begin{bmatrix} \overline{C} \\ \overline{D} \end{bmatrix} = \begin{bmatrix} 1 + B \\ B \end{bmatrix}, \] so \(\overline{C} = \frac{1 + B}{3}\) and \(\overline{D} = \frac{B}{2}\). (c) The normal equations with \(b = (1, 0, 0)\) are \[ \begin{bmatrix} 3 & 0 \\ 0 & 2 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} 1 \\ 1 \end{bmatrix}, \] so \[ p = A\bar{x} = \begin{bmatrix} 1 & -1 \\ 1 & 0 \\ 1 & 1 \end{bmatrix} \begin{bmatrix} 1/3 \\ 1/2 \end{bmatrix} = \begin{bmatrix} -1/6 \\ 1/3 \\ 5/6 \end{bmatrix}. \] (d) The columns of \(A\) are already orthogonal (the first column is \((1, 1, 1)\), the second is \((-1, 0, 1)\)), so Gram-Schmidt just normalizes them: \[ Q = \begin{bmatrix} 1/\sqrt{3} & -1/\sqrt{2} \\ 1/\sqrt{3} & 0 \\ 1/\sqrt{3} & 1/\sqrt{2} \end{bmatrix}. \]

Keywords: least squares, normal equations, projection, Gram-Schmidt