Measurements are collected sequentially, either in batches or one-by-one. After collection of a measurement y corresponding to the parameter x, the surrogate model is trained on the dataset of all available observations, D = defines the set of parameters expected to provide maximal utility, and corresponds to the parameter recommended for subsequent measurement. Typically, this iterative procedure is repeated until a pre-defined stopping criterion, such as the exhaustion of an experimental budget, is met. Algorithm 1 shows pseudocode for a BO loop, and Figure visualizes the procedure. The first set of parameters is often not recommended by BO, but are rather produced by random sampling, a low-discrepancy sequence, or an experimental design strategy. This is known as the initial design phase.