As we know, Gaussian processes modeling is often refer to as nonparametric modeling. But why? It has parameters in its covariance kernel:
\begin{align*} K(x_i,x_j) = h^2\text{exp}\left(\frac{-(x_i-x_j)^2}{\lambda^2}\right) \end{align*}
From the example of Gaussian kernel above, $h$ and $\lambda$ are the hyperparameters.

### Some Points:

• “Parametric” versus “Nonparametric” are terms that do not apply to particular processes, they apply to the entire family of processes that could be fit to data.
• So if there’s no limit tothe number of parameters, we are estimating the parameter of infinite dimension, it turns out to be a nonparametric problem.
• Models can be nonparametric in one sense and parametric in another (e.g. semiparametric models).
• Nonparametric just means we cannot describe the model by a fixed set of parameters but they still can have parameters.
• Return back to the example above, $h$ and $\lambda$ are not parameters because they do not represent anything physical about the data itself. They just parametrize the covariance.

### Relevant Materials:

 Anna Scaife: Machine Learning: Gaussian Process Modelling in Python [lecture]
 Yee Whye Teh: Bayesian Nonparametric [lecture]

0%