Olympus' fully-categorical datasets comprise a discrete set of possible parameter settings for which a corresponding set of measurement(s) is available. We use Bayesian neural networks (BNNs) to capture heteroscedastic aleatoric (datainherent) uncertainties associated with measurements such that users of Olympus can sample noisy realizations of previous experiments in their optimization benchmarks. The BNN models are trained with variational inference. Variational learning finds the parameters θ of a distribution on BNN weights q θ (w) which best resembles the true Bayesian posterior on the weights. This procedure provides the predictive distribution for the output y * given an unseen input instance x * ,