Illustration of a Gaussian process (GP) model with an RBF kernel over a toy one-dimensional NPM feature space. The black points are the observed data from a toy structure-property relationship, f (x). The blue line and shaded region visualize the GP model trained on the observed data: the line is the approximation ŷ (x) of the structure-property relationship, while the shaded region illustrates the uncertainty by covering ŷ (x) ± 2σ(x). The GP model shows large (small) uncertainty in regions of feature space far from (close to) the observations. Hyperparameters of GPs. GPs are non-parametric models, but most useful kernel functions used in GPs contain hyperparameters. For example, the RBF kernel in eqn. has the length-scale γ and the signal variance σ f hyperparameters. To learn the hyperparameters of the kernel that give us the best approximation of f (x), we could perform a grid search over hyperparameter space for those that minimize the validation loss. A more popular and faster way to estimate hyperparameters of the kernel is to maximize the marginal likelihood of the observed data as a function of the kernel hyperparameters. [ ] Generally, at each iteration of BO, the hyperparameters of the kernel are updated to account for the newly acquired observation.