are referred to as a "directed divergences" between two distributions; Kullback preferred the term discrimination information. The term "divergence" is in contrast to a distance (metric), since the symmetrized divergence does not satisfy the triangle inequality. Numerous references to earlier uses of the symmetrized divergence and to other statistical distances are given in . The asymmetric "directed divergence" has come to be known as the Kullback–Leibler divergence, while the symmetrized "divergence" is now referred to as the Jeffreys divergence. Definition For discrete probability distributions and defined on the same sample space, , the relative entropy from to is defined to be which is equivalent to In other words, it is the expectation of the logarithmic difference between the probabilities and , where the expectation is taken using the probabilities . Relative entropy is defined so only if for all , implies (absolute continuity). Else it is often defined as , but the value is possible even if everywhere, provided that is infinite. Analogous comments apply to the continuous and general measure cases defined below. Whenever is zero the contribution of the corresponding term is interpreted as zero because For distributions and of a continuous random variable, relative entropy is defined to be the integral: where and denote the probability densities of and . More generally, if and are probability measures on a measurable space , and is absolutely continuous with respect to , then the relative entropy from to is defined as where is the Radon–Nikodym derivative of with respect to ,ie. the unique -almost everywhere defined function on such that which exists because is absolutely continuous with respect to . Also we assume the expression on the right-hand side exists. Equivalently (by the chain rule), this can be written as which is the entropy of relative to . Continuing in this case, if is any measure on for which densities and with and exist (meaning that and are both absolutely continuous with respect to ), then the relative entropy from to is given as Note that such a measure for which densities can be defined always exists, since one can take although in practice it will usually be one that in the context like counting measure for discrete distributions, or Lebesgue measure or a convenient variant thereof like Gaussian measure or the uniform measure on the sphere, Haar measure on a Lie group etc. for continuous distributions. The logarithms