maximise the expected relative entropy between the prior and the posterior. When posteriors are approximated to be Gaussian distributions, a design maximising the expected relative entropy is called Bayes d-optimal. Discrimination information Relative entropy can also be interpreted as the expected discrimination information for over : the mean information per sample for discriminating in favor of a hypothesis against a hypothesis , when hypothesis is true. Another name for this quantity, given to it by I. J. Good, is the expected weight of evidence for over to be expected from each sample. The expected weight of evidence for over is not the same as the information gain expected per sample about the probability distribution of the hypotheses, Either of the two quantities can be used as a utility function in Bayesian experimental design, to choose an optimal next question to investigate: but they will in general lead to rather different experimental strategies. On the entropy scale of information gain there is very little difference between near certainty and absolute certainty—coding according to a near certainty requires hardly any more bits than coding according to an absolute certainty. On the other hand, on the logit scale implied by weight of evidence, the difference between the two is enormous – infinite perhaps; this might reflect the difference between being almost sure (on a probabilistic level) that, say, the Riemann hypothesis is correct, compared to being certain that it is correct because one has a mathematical proof. These two different scales of loss function for uncertainty are both useful, according to how well each reflects the particular circumstances of the problem in question. Principle of minimum discrimination information The idea of relative entropy as discrimination information led Kullback to propose the Principle of (MDI): given new facts, a new distribution should be chosen which is as hard to discriminate from the original distribution as possible; so that the new data produces as small an information gain as possible. For example, if one had a prior distribution over and , and subsequently learnt the true distribution of was , then the relative entropy between the new joint distribution for and , , and the earlier prior distribution would be: i.e. the sum of the relative entropy of the prior distribution for from the updated distribution , plus the expected value (using the probability distribution ) of the relative entropy of the prior conditional distribution