Curve fitting

Generally, we will have some datapoint pairs \(x_i, y_i\). We will have some function \(y\) with parameters \(a\) that will be evaluated at a point \(x\): \(y(a, x)\).

Our goal is to find the set of parameters \(a\) that gives us the best fit of the data. Commonly, this implies minimizing the squared error between the prediction and the data,

\[y(a,x_i)-y_i = r_i \]

where \(r\) is the residual vector. The least squares fit is formulated as finding \(a\) so as to minimize \(\| r\|\).

In general, this is an optimization problem (much more complicated than you would expect!) since \(y(a,x)\) can be complex.