Skip to content

4.3-exam2-2010

Quiz 2, 2010 · Section 4.3: Least Squares Approximations Source: exam2_2.pdf, pp. 4-5

Problem

2. (34 points)

(a) \(p = A\hat{x}\) is the vector in \(C(A)\) nearest to a given vector \(b\). If \(A\) has independent columns, what equation determines \(\hat{x}\)? What are all the vectors perpendicular to the error \(e = b - A\hat{x}\)? What goes wrong if the columns of \(A\) are dependent?

(b) Suppose \(A = QR\) where \(Q\) has orthonormal columns and \(R\) is upper triangular invertible. Find \(\hat{x}\) and \(p\) in terms of \(Q\) and \(R\) and \(b\) (not \(A\)).

(c) If \(q_1\) and \(q_2\) are any orthonormal vectors in \(R^5\), give a formula for the projection \(p\) of any vector \(b\) onto the plane spanned by \(q_1\) and \(q_2\) (write \(p\) as a combination of \(q_1\) and \(q_2\)).

Solution (a) \(\hat{x}\) is determined by the equation \(\hat{x} = (A^TA)^{-1}A^Tb\) (since \(A\) has independent columns, \(A^TA\) is invertible whether or not \(A\) is square). The vectors perpendicular to an arbitrary error vector are the elements of the column space of \(A\). If the columns of \(A\) are dependent, \(A^TA\) is no longer invertible, and there is no unique nearest vector (i.e., there are multiple solutions). RUBRIC: 4 points for the determining equation (1 point off for actually inverting \(A^TA\) or saying that it was invertible), 3 points for identifying the column space, and three points for identifying the multiple solutions (1 point off if you just say that \(A^TA\) is not invertible). Note that you cannot write \(A^{-1}B\) as \(\frac{B}{A}\): this only works for numbers because multiplication and division are commutative, which is not true for matrices. (b) Since \(Q^TQ = I\) and \(R\) is invertible, we obtain \[ \hat{x} = (A^TA)^{-1}A^Tb = ((QR)^T(QR))^{-1}(QR)^Tb = (R^TQ^TQR)^{-1}R^TQ^Tb = R^{-1}(R^T)^{-1}R^TQ^Tb = R^{-1}Q^Tb \] and \[ p = (QR)\hat{x} = QQ^Tb. \] Note that \(QQ^T\) is not the identity matrix in general. RUBRIC: 6 points for finding \(\hat{x}\), 4 points for \(p\). One point off from each if the equations are not simplified, more points off for bad form, having variables other than \(Q\), \(R\) and \(b\), etc. (c) \(p = (q_1^Tb)q_1 + (q_2^Tb)q_2\). RUBRIC: little partial credit. If you identified the difference between \(b\) and \(p\) instead, you may have gotten some points.

Keywords: least squares, normal equations, QR decomposition, projection