Computing the in-sample predictions once all data has been collected for a given optimization campaign provides an alternative method to compare performance across different search spaces, especially when evaluating an objective function repeatedly is not feasible. To elaborate, a single observation is unlikely to be the same as the mean response if the same parameter set were tried multiple times. Rather than choose the sample with the best observed data, an alternative is to take a 11 In-sample predictions (meaning predictions for trials that completed) are used rather than the raw observed data due to noise in the latter.