6.3-final-2006¶
Final Exam, 2006 · Section 6.3: Symmetric Positive Definite Matrices Source: finalexam_1.pdf, p. 145
Problem¶
5 (5+5=10 pts.) If \(A\) is 3 by 3 symmetric positive definite, then \(Aq_i = \lambda_iq_i\) with positive eigenvalues and orthonormal eigenvectors \(q_i\).
Suppose \(x = c_1q_1 + c_2q_2 + c_3q_3\).
(a) Compute \(x^Tx\) and also \(x^TAx\) in terms of the \(c\)'s and \(\lambda\)'s.
(b) Looking at the ratio of \(x^TAx\) in part (a) divided by \(x^Tx\) in part (a), what \(c\)'s would make that ratio as large as possible? You can assume \(\lambda_1 < \lambda_2 < \dots < \lambda_n\). Conclusion: the ratio \(x^TAx/x^Tx\) is a maximum when \(x\) is __.
Solution
(a) \[ \begin{aligned} x^Tx &= (c_1q_1^T + c_2q_2^T + c_3q_3^T)(c_1q_1 + c_2q_2 + c_3q_3) \\ &= c_1^2q_1^Tq_1 + c_1c_2q_1^Tq_2 + \dots + c_3c_2q_3^Tq_2 + c_3^2q_3^Tq_3 \\ &= c_1^2 + c_2^2 + c_3^2. \end{aligned} \] \[ \begin{aligned} x^TAx &= (c_1q_1^T + c_2q_2^T + c_3q_3^T)(c_1Aq_1 + c_2Aq_2 + c_3Aq_3) \\ &= (c_1q_1^T + c_2q_2^T + c_3q_3^T)(c_1\lambda_1q_1 + c_2\lambda_2q_2 + c_3\lambda_3q_3) \\ &= c_1^2\lambda_1q_1^Tq_1 + c_1c_2\lambda_2q_1^Tq_2 + \dots + c_3c_2\lambda_2q_3^Tq_2 + c_3^2\lambda_3q_3^Tq_3 \\ &= c_1^2\lambda_1 + c_2^2\lambda_2 + c_3^2\lambda_3. \end{aligned} \] (b) We maximize \((c_1^2\lambda_1 + c_2^2\lambda_2 + c_3^2\lambda_3)/(c_1^2 + c_2^2 + c_3^2)\) when \(c_1 = c_2 = 0\), so \(x = c_3q_3\) is a multiple of the eigenvector \(q_3\) with the largest eigenvalue \(\lambda_3\). (Also notice that the maximum value of this "Rayleigh quotient" \(x^TAx/x^Tx\) is the largest eigenvalue itself. This is another way of finding eigenvectors: maximize \(x^TAx/x^Tx\) numerically.)Keywords: positive definite, Rayleigh quotient, eigenvectors, orthonormal