Skip to content

4.4-exam2-2012-strang

Quiz 2, 2012 (strang) · Section 4.4: Orthonormal Bases and Gram-Schmidt Source: exam2_2.pdf, pp. 35-36

Problem

3 (30 pts.)

(a) Starting from independent vectors \(a_1\) and \(a_2\), use Gram-Schmidt to find formulas for two orthonormal vectors \(q_1\) and \(q_2\) (combinations of \(a_1\) and \(a_2\)).

(b) The connection between the matrices \(A = [a_1 \ a_2]\) and \(Q = [q_1 \ q_2]\) is often written \(A = QR\). From your answer to Part (a), what are the entries in this matrix \(R\)?

(c) The least squares solution \(\widehat{x}\) to the equation \(Ax = b\) comes from solving what equation? If \(A = QR\) as above, show that \(R\widehat{x} = Q^Tb\).

Solution (a) \[ q_1 = \frac{a_1}{\|a_1\|}, \qquad q_2 = \frac{a_2 - (a_2^Tq_1)q_1}{\|a_2 - (a_2^Tq_1)q_1\|} = \left( a_2 - \frac{a_2^Ta_1}{a_1^Ta_1}a_1 \right) \Big/ \left\| a_2 - \frac{a_2^Ta_1}{a_1^Ta_1}a_1 \right\|. \] (b) Re-arranging the expressions above gives \[ a_1 = q_1\|a_1\|, \qquad a_2 = (a_2^Tq_1)q_1 + \|a_2 - (a_2^Tq_1)q_1\|q_2, \] and thus \[ R = \begin{bmatrix} a_1^Tq_1 & a_2^Tq_1 \\ a_1^Tq_2 & a_2^Tq_2 \end{bmatrix} = \begin{bmatrix} \|a_1\| & a_2^Tq_1 \\ 0 & \|a_2 - (a_2^Tq_1)q_1\| \end{bmatrix}. \] (c) \(\widehat{x}\) comes from solving \(A^TA\widehat{x} = A^Tb\). Suppose we have \(A = QR\). Notice that \(Q^TQ = I\), so \[ A^TA = (QR)^T(QR) = R^TQ^TQR = R^TR, \] and as \(a_1\) and \(a_2\) are independent, \(R\) is invertible; thus \(R^T\) is also invertible. Hence \[ A^TA\widehat{x} = A^Tb \iff R^TR\widehat{x} = R^TQ^Tb \iff R\widehat{x} = Q^Tb. \]

Keywords: Gram-Schmidt, orthonormal basis, QR decomposition, least squares