Skip to content

4.2-exam2-2013-strang-b

Quiz 2, 2013 (strang) · Section 4.2: Projections onto Lines and Subspaces Source: exam2_2.pdf, pp. 67, 70-71

Problem

3 (30 pts.)

This 4 by 4 Hadamard matrix is an orthogonal matrix. Its columns are orthogonal unit vectors.

\[ Q = \frac{1}{2} \begin{bmatrix} 1 & 1 & 1 & 1 \\ 1 & -1 & 1 & -1 \\ 1 & 1 & -1 & -1 \\ 1 & -1 & -1 & 1 \end{bmatrix} = \begin{bmatrix} q_1 & q_2 & q_3 & q_4 \end{bmatrix} \]

(a) What projection matrix \(P_4\) (give numbers) will project every \(b\) in \(R^4\) onto the line through \(q_4\)?

(b) What projection matrix \(P_{123}\) will project every \(b\) in \(R^4\) onto the subspace spanned by \(q_1\), \(q_2\), and \(q_3\)? Remember that those columns are orthogonal.

(c) Suppose \(A\) is the 4 by 3 matrix whose columns are \(q_1\), \(q_2\), \(q_3\). Find the least-squares solution \(\widehat{x}\) to the four equations

\[ Ax = \frac{1}{2} \begin{bmatrix} 1 & 1 & 1 \\ 1 & -1 & 1 \\ 1 & 1 & -1 \\ 1 & -1 & -1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} = \begin{bmatrix} 1 \\ 2 \\ 3 \\ 4 \end{bmatrix} = b. \]

What is the error vector \(e\)?

Solution (a) Letting \[ A = \begin{bmatrix} 1 \\ -1 \\ -1 \\ 1 \end{bmatrix}, \] the projection matrix that projects every \(b \in R^4\) onto the column space of \(A\) (which is the line through \(q_4\)) is given by the formula \[ A(A^TA)^{-1}A^T = \begin{bmatrix} 1 \\ -1 \\ -1 \\ 1 \end{bmatrix} \left( \begin{bmatrix} 1 & -1 & -1 & 1 \end{bmatrix} \begin{bmatrix} 1 \\ -1 \\ -1 \\ 1 \end{bmatrix} \right)^{-1} \begin{bmatrix} 1 & -1 & -1 & 1 \end{bmatrix} \] \[ = \frac{1}{4} \begin{bmatrix} 1 \\ -1 \\ -1 \\ 1 \end{bmatrix} \begin{bmatrix} 1 & -1 & -1 & 1 \end{bmatrix} \] \[ = \frac{1}{4} \begin{bmatrix} 1 & -1 & -1 & 1 \\ -1 & 1 & 1 & -1 \\ -1 & 1 & 1 & -1 \\ 1 & -1 & -1 & 1 \end{bmatrix}. \] (b) Letting \[ A = \begin{bmatrix} 1 & 1 & 1 \\ 1 & -1 & 1 \\ 1 & 1 & -1 \\ 1 & -1 & -1 \end{bmatrix}, \] the projection matrix that projects every \(b \in R^4\) onto the column space of \(A\) (which is the subspace spanned by \(q_1, q_2\) and \(q_3\)) is given by the formula \[ A(A^TA)^{-1}A^T = \frac{1}{4} \begin{bmatrix} 1 & 1 & 1 \\ 1 & -1 & 1 \\ 1 & 1 & -1 \\ 1 & -1 & -1 \end{bmatrix} \begin{bmatrix} 1 & 1 & 1 & 1 \\ 1 & -1 & 1 & -1 \\ 1 & 1 & -1 & -1 \end{bmatrix} \] \[ = \frac{1}{4} \begin{bmatrix} 3 & 1 & 1 & -1 \\ 1 & 3 & -1 & 1 \\ 1 & -1 & 3 & 1 \\ -1 & 1 & 1 & 3 \end{bmatrix}. \] (c) We must solve the new system \[ A^TA\hat{x} = A^Tb. \] Since \(A^TA = I\), we have \[ \hat{x} = A^Tb = \begin{bmatrix} 5 \\ -1 \\ -2 \end{bmatrix}. \] Then \(A\hat{x} = \begin{bmatrix} 1 \\ 2 \\ 3 \\ 4 \end{bmatrix}\), and \(e = b - A\hat{x} = 0\).

Keywords: Hadamard matrix, projection matrix, orthogonal columns, least squares