For single-task Bayesian optimization, we leverage a Gaussian process (GP) as the probabilistic model in BO due to its excellent performance in the limit of small of data. A GP is a stochastic process characterized by a mean µ ! (𝑥) and covariance function 𝑘 ! (𝑥, 𝑥 " ). The covariance function is often called a kernel, which is the term we will use henceforth.