While metrics are symmetric and generalize linear distance, satisfying the triangle inequality, divergences are asymmetric in general and generalize squared distance, in some cases satisfying a generalized Pythagorean theorem. In general does not equal , and while this can be symmetrized (see ), the asymmetry is an important part of the geometry. It generates a topology on the space of probability distributions. More concretely, if is a sequence of distributions such that then it is said that Pinsker's inequality entails that where the latter stands for the usual convergence in total variation. Fisher information metric Relative entropy is directly related to the Fisher information metric. This can be made explicit as follows. Assume that the probability distributions and are both parameterized by some (possibly multi-dimensional) parameter . Consider then two close by values of and so that the parameter differs by only a small amount from the parameter value . Specifically, up to first order one has (using the Einstein summation convention) with a small change of in the direction, and the corresponding rate of change in the probability distribution. Since relative entropy has an absolute minimum 0 for , i.e. , it changes only to second order in the small parameters . More formally, as for any minimum, the first derivatives of the divergence vanish and by the Taylor expansion one has up to second order where the Hessian matrix of the divergence must be positive semidefinite. Letting vary (and dropping the subindex 0) the Hessian defines a (possibly degenerate) Riemannian metric on the parameter space, called the Fisher information metric. Fisher information metric theorem When satisfies the following regularity conditions: exist, where is independent of then: Variation of information Another information-theoretic metric is variation of information, which is roughly a symmetrization of conditional entropy. It is a metric on the set of partitions of a discrete probability space. Relation to other quantities of information theory Many of the other quantities of information theory can be interpreted as applications of relative entropy to specific cases. Self-information The self-information, also known as the information content of a signal, random variable, or event is defined as the negative logarithm of the probability of the given outcome occurring. When applied to a discrete random variable, the self-information can be represented as is the relative entropy of the probability distribution from a Kronecker delta representing certainty that — i.e. the number of extra