4.3-final-2002¶
Final Exam, 2002 · Section 4.3: Least Squares Approximations Source: finalexam_1.pdf, pp. 86, 92-93
Problem¶
5 Suppose the 4 by 2 matrix \(Q\) has orthonormal columns.
(a) Find the least squares solution \(\widehat{x}\) to \(Qx = b\).
(b) Explain why \(QQ^{T}\) is not positive definite.
(c) What are the (nonzero) singular values of \(Q\), and why?
Solution
(a) The least squares solution is \[ \widehat{x} = (Q^TQ)^{-1}Q^Tb = I^{-1}Q^Tb = Q^Tb. \] (b) One approach: \(Q^T\) is 2 by 4, so \(Q^Tx = 0\) has free variables and many nonzero solutions. Then \(QQ^Tx = 0\) too, so \(QQ^T\) is singular and not positive definite. Second approach: rank \(Q = 2\), so rank \(QQ^T \leq 2\); as a 4 by 4 matrix it has a 2-dimensional nullspace, hence two zero eigenvalues, and is not positive definite. (c) The singular values of \(Q\) are the square roots of the eigenvalues of \(Q^TQ = I\), namely all 1's.Keywords: least squares, orthonormal columns, positive definite, singular values