Skip to content

4.3-exam2-1997-strang

Quiz 2, 1997 (strang) · Section 4.3: Least Squares Approximations Source: exam2_1.pdf, pp. 9, 10-11

Problem

2. At \(t = 1, 2, 3\) we are given values \(b_1, b_2, b_3\). The idea is to fit the best straight line \(b = C + Dt\) to those three points.

(a) What three equations in two unknowns will have a solution if the three points lie exactly on a line?

(b) Under what condition \(m b_1 + n b_2 + p b_3 = 0\) (find \(m, n, p\)) will the three points lie on a line? (You could use elimination or your answer to Question 1(a).)

(c) Find the best line \(\bar{C} + \bar{D}t\) if the values are \((b_1, b_2, b_3) = (0, 0, 1)\).

(d) What \(3 \times 3\) matrix \(P\) projects every vector onto the plane containing the column vectors \((1, 1, 1)\) and \((1, 2, 3)\)?

Solution 2a. The three equations are \[ \begin{aligned} C + D &= b_1 \\ C + 2D &= b_2 \\ C + 3D &= b_3 \end{aligned} \qquad A = \begin{bmatrix} 1 & 1 \\ 1 & 2 \\ 1 & 3 \end{bmatrix}. \] 2b. Elimination (subtracting equation 1) gives \(D = b_2 - b_1\) and \(2D = b_3 - b_1\), so \[ 0 = (b_3 - b_1) - 2(b_2 - b_1), \qquad 0 = b_3 - 2b_2 + b_1. \] Thus \((m, n, p) = (1, -2, 1)\). Other method: a basis for the left nullspace of \(A\) is \(\begin{bmatrix} 1 \\ -2 \\ 1 \end{bmatrix}\) since \(\begin{bmatrix} 1 & 1 & 1 \\ 1 & 2 & 3 \end{bmatrix} \begin{bmatrix} 1 \\ -2 \\ 1 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}\); by Question 1(a), \(b\) must be orthogonal to this vector, which means \(b_1 - 2b_2 + b_3 = 0\). 2c. \[ A^T A = \begin{bmatrix} 3 & 6 \\ 6 & 14 \end{bmatrix}, \qquad A^T b = \begin{bmatrix} 1 \\ 3 \end{bmatrix}. \] Solving \[ \begin{aligned} 3\bar{C} + 6\bar{D} &= 1 \\ 6\bar{C} + 14\bar{D} &= 3 \end{aligned} \] gives \(\bar{D} = \frac{1}{2}\) and \(\bar{C} = -\frac{2}{3}\). 2d. \[ \begin{aligned} P &= A(A^T A)^{-1} A^T \\ &= \begin{bmatrix} 1 & 1 \\ 1 & 2 \\ 1 & 3 \end{bmatrix} \frac{\begin{bmatrix} 14 & -6 \\ -6 & 3 \end{bmatrix}}{6} \begin{bmatrix} 1 & 1 & 1 \\ 1 & 2 & 3 \end{bmatrix} \\ &= \begin{bmatrix} 1 & 1 \\ 1 & 2 \\ 1 & 3 \end{bmatrix} \frac{1}{6} \begin{bmatrix} 8 & 2 & -4 \\ -3 & 0 & 3 \end{bmatrix} \\ &= \frac{1}{6} \begin{bmatrix} 5 & 2 & -1 \\ 2 & 2 & 2 \\ -1 & 2 & 5 \end{bmatrix}, \end{aligned} \] and \(\begin{bmatrix} 1 \\ -2 \\ 1 \end{bmatrix}\) is still in the nullspace of \(P\), as it must be.

Keywords: least squares, normal equations, left nullspace, projection matrix