Skip to content

4.3-exam2-2014-strang

Exam II, 2014 (strang) · Section 4.3: Least Squares Approximations Source: exam2_2.pdf, pp. 85, 90

Problem

2. (25 points total)

(a) (11 points) Suppose \(\widehat{x}\) is the best least squares solution to \(Ax = b\) and \(\widehat{y}\) is the best least squares solution to \(Ay = c\).

Does this tell you the best least squares solution \(\widehat{z}\) to \(Az = b + c\)? If so, what is the best \(\widehat{z}\) and why?

(b) (7 points) If \(Q\) is an \(m\) by \(n\) matrix with orthonormal columns, find the best least squares solution \(\widehat{x}\) to \(Qx = b\).

(c) (7 points) If \(A = QR\), where \(R\) is square invertible and \(Q\) is the same as in (b), find the least squares solution to \(Ax = b\).

Solution (a) Denote by \(P\) the projection onto the column space of \(A\). We have \(A\hat{x} = Pb\) and \(A\hat{y} = Pc\). That means \(A\hat{x} + A\hat{y} = Pb + Pc = P(b + c)\). It follows that \(\hat{x} + \hat{y}\) is the least squares solution for \(A\hat{z} = b + c\). (b) The least squares solution can be written as \(\hat{x} = (Q^TQ)^{-1}Q^Tb\). As \(Q\) is orthonormal, \(Q^TQ = I\). Therefore, \(\hat{x} = Q^Tb\). Alternatively, solving least squares means finding a solution to \(Q^TQ\hat{x} = Q^Tb\). As \(Q^TQ = I\), we see that \(\hat{x} = Q^Tb\). (c) The least squares solution can be written as \(\hat{x} = (A^TA)^{-1}A^Tb = (R^TQ^TQR)^{-1}R^TQ^Tb\). As \(Q\) is orthonormal, \(Q^TQ = I\). Therefore, \(\hat{x} = (R^TR)^{-1}R^TQ^Tb\). As \(R\) is invertible, we get \[ \hat{x} = (R^TR)^{-1}R^TQ^Tb = R^{-1}(R^T)^{-1}R^TQ^Tb = R^{-1}Q^Tb. \]

Keywords: least squares, orthonormal columns, QR decomposition