• Expected Predictive Information Gain (EPIG) measures, for each x ∈ U r , its expected reduction in predictive uncertainty at a randomly sampled x ⋆ from a target input distribution p ⋆ (x ⋆ ). Mathematically, EPIG can be formulated as the expected mutual information between y and y ⋆ given x and x ⋆ , which can be written as an expected KL divergence between p(y, y ⋆ | x, x ⋆ ) and p(y | x)p(y ⋆ | x ⋆ ):