Fitting polynomials #
We have learned that any continuous function can be approximated by a polynomial. Suppose that we have gathered a data set:
\[\mathcal{D} = \{ (\vec{x_i}, \vec{y}_i) \}_{i \in I} \]
where \(\vec{x}_i\) is the feature vector for the \(i\)th data point and \(\vec{y}_i\) is the coresponding prediction vector. The fundamental questions of this section are:
Questions:
- How does one go about finding a polynomial function \(f\) so that \[ f(\vec{x}_i) \approx \vec{y}_i \] for all \(i \in I\); that is, the function fits the data \(\mathcal{D}\)?
- Under what circumstances will this function not only fit the data \(\mathcal{D}\), but also make good predictions on yet unobserved data?
- Do our methods extend to other families of functions in addition to polynomials?
We will introduce methods and ideas that are important not only for polynomial approximation, but also other machine learning methods that we will discuss later in the course.