curves. Note that these elbow regions are not sharply defined. For gentle refinement at θ = 10, χ 2 has drastically decreased while S KL remains relatively small. Overfitting and underfitting are illustrated by the double-well system (see Figure ). We have underfitting for θ = 100 as the calculated average is multiple standard deviations away from the experimental value. We have overfitting for θ = 1 as the calculated average agrees nearly perfectly with the experimental average despite the error. Both extreme cases are unlikely with respect to the Gaussian likelihood, Eq. ( ). The character of the distribution of the observable can change quite substantially even when the optimal entropy values appear relatively small, as we discuss in more detail below. For the optimal ensembles, the statistical weights of the left state [with observable values y(x) = x < 0] are given by 42%, 23%, 12% for (θ, S KL ) = (100, 0.01), (10, 0.17), and