4.3-exam2-2007-johnson-b¶
Quiz 1, 2007 (johnson) · Section 4.3: Least Squares Approximations Source: exam2_1.pdf, pp. 152-153
Problem¶
3 (12 pts.) Construct an example of a least-square curve-fitting problem where the solution (the least-square fit parameters) is not unique. (You need not solve it, just write down the \(Ax = b\) equations that you would solve by least-squares to minimize \(\|Ax - b\|^2\).)
Solution
The solution is not unique if the matrix \(A\) is not of full column rank. (Problem 1(c) is an example of this type.) There are many possible examples. For example, you could have more unknowns than data points, e.g. you could be fitting to a line \(C + Dt\) but only have a single point \((t_1, b_1)\)--obviously, a single point is not enough to determine a line uniquely. In terms of matrices, \(A = (1 \;\; t_1)\), \(x = (C, D)^T\), and \(b = (b_1)\), which obviously does not have full column rank: there are more columns than rows in \(A\)! Alternatively, you could be fitting a line \(C + Dt\) to multiple points \((t_1, b_1), (t_1, b_2), (t_1, b_3)\) etcetera with the same \(t\) coordinate--this is enough information to determine \(C\) or \(D\) but not both. In this case you get a matrix equation of the form: \[ \begin{bmatrix} 1 & t_1 \\ 1 & t_1 \\ \vdots & \vdots \\ 1 & t_1 \end{bmatrix} \begin{bmatrix} C \\ D \end{bmatrix} = \begin{bmatrix} b_1 \\ b_2 \\ \vdots \\ b_m \end{bmatrix}, \] which obviously does not have full column rank: column 2 \(= t_1 \cdot\) column 1 in \(A\). You could also construct an example where your fit parameters are not really independent, regardless of the data. For example, if you are fitting to \(C + Dt + E(3t - 1)\), in which case \(E\) does not add any information because \(3t - 1\) is a linear combination of \(C\) and \(Dt\). Correspondingly, the third column of \(A\) will be three times the second column minus the first column.Keywords: least squares, curve fitting, uniqueness, column rank