(xpredictive mean (xpredictive mean - - - - - - figure gp regression of synthetic data set with bandwidth (leftand (rightthe black dots represent the data and the blue curve is the latent function ( sin( pxthe red curve is the mean of the gp predictive distribution given by ( )and the shaded region is the confidence bandcorresponding to the predictive variance given in ( empirical bayes one approach to setting the hyperparameter th is to determine its posterior (th yand obtain point estimatefor instance via its maximum posteriori estimate howeverthis can be computationally demanding exercise what is frequently done in practice is to consider instead the marginal likelihood ( thand maximize this with respect to th this procedure is called empirical bayes considering again the mean function to be identically zerofrom ( )we have that ( this multivariate normal with mean and covariance matrix in immediately giving an expression for the marginal log-likelihood ( ln ( thln( pln det( )yk- we notice that only the second and third terms in ( depend on th considering partial derivative of ( with respect to single element th of the hyperparameter vector th yields # - - ln ( thtr ky ky - ( yth th th where th is the element-wise derivative of matrix with respect to th if these partial derivatives can be computed for each hyperparameter thgradient information could be used when maximizing ( example (gp regression (cont )continuing example we plot in figure the marginal log-likelihood as function of the noise level and bandwidth parameter the maximum is attained for bandwidth parameter around and which is very close to the left panel of figure for the case where was assumed to be known (and equal to we note here that the marginal log-likelihood is extremely flatperhaps owing to the small number of points