Skip to content

4.3-exam2-2009

Quiz 2, 2009 · Section 4.3: Least Squares Approximations Source: exam2_1.pdf, pp. 168-169

Problem

Problem 3 (20 points = 10 + 5 + 5): You are told that the least-square linear fit to three points \((0, b_1)\), \((1, b_2)\), and \((2, b_3)\) is \(C + Dt\) for \(C = 1\) and \(D = -2\). That is, the fit is \(1 - 2t\). In this question, you will work backwards from this fit to reason about the unknown values \(b = (b_1 \; b_2 \; b_3)^T\) at the coordinates \(t = 0, 1, 2\).

(i) Write down the explicit equations that \(b\) must satisfy for \(1 - 2t\) to be the least-square linear fit. (The points do not have to fall exactly on the line.)

(ii) If all the points fall exactly on the line \(1 - 2t\), then \(b = \_\_\_\_\_\_\). Check that this satisfies your equations in (i).

(iii) More generally, if all the points fall exactly on any straight line, then \(b\) is in the ______ space of what matrix? (Write down the matrix.)

Solution Answers: (i) see below; (ii) \(b = (1, -1, -3)\); (iii) see below. (i) The system that we would solve if the line passed exactly through all of the points is \[ \begin{bmatrix} 1 & 0 \\ 1 & 1 \\ 1 & 2 \end{bmatrix} \begin{bmatrix} C \\ D \end{bmatrix} = \begin{bmatrix} b_1 \\ b_2 \\ b_3 \end{bmatrix}. \] Since the line may not pass through all the points, this system may have no solution, and instead we find the least-square solution by solving the normal equations \(A^T A \hat{x} = A^T b\): \[ \begin{bmatrix} 3 & 3 \\ 3 & 5 \end{bmatrix} \begin{bmatrix} C \\ D \end{bmatrix} = \begin{bmatrix} b_1 + b_2 + b_3 \\ b_2 + 2b_3 \end{bmatrix}. \] Since the least-square fit is \(1 - 2t\), the above linear system has solution \((1, -2)^T\). Hence \(b_1, b_2, b_3\) should satisfy \[ b_1 + b_2 + b_3 = -3, \qquad b_2 + 2b_3 = -7. \] (ii) If all the points fall exactly on the line \(1 - 2t\), then \(b_1 = 1 - 2 \cdot 0 = 1\), \(b_2 = 1 - 2 \cdot 1 = -1\), \(b_3 = 1 - 2 \cdot 2 = -3\). We plug this back into the relations above and check: \[ 1 - 1 - 3 = -3, \qquad -1 + 2 \times (-3) = -7. \] (iii) If all points fall exactly on a straight line, the following system would have a solution: \[ \begin{bmatrix} 1 & 0 \\ 1 & 1 \\ 1 & 2 \end{bmatrix} \begin{bmatrix} C \\ D \end{bmatrix} = \begin{bmatrix} b_1 \\ b_2 \\ b_3 \end{bmatrix}. \] In other words, \(b\) lies in the column space of the matrix \(\begin{bmatrix} 1 & 0 \\ 1 & 1 \\ 1 & 2 \end{bmatrix}\).

Keywords: least squares, linear fit, normal equations, column space