mutual information of partitions of and as these partitions become finer and finer. Thus it is invariant under non-linear homeomorphisms (continuous and uniquely invertible maps), including linear transformations of and , and still represents the amount of discrete information that can be transmitted over a channel that admits a continuous space of values. For the direct analogue of discrete entropy extended to the continuous space, see limiting density of discrete points. Properties of differential entropy For probability densities and , the Kullback–Leibler divergence is greater than or equal to 0 with equality only if almost everywhere. Similarly, for two random variables and , and with equality if and only if and are independent. The chain rule for differential entropy holds as in the discrete case . Differential entropy is translation invariant, i.e. for a constant . Differential entropy is in general not invariant under arbitrary invertible maps. In particular, for a constant For a vector valued random variable and an invertible (square) matrix In general, for a transformation from a random vector to another random vector with same dimension , the corresponding entropies are related via where is the Jacobian of the transformation . The above inequality becomes an equality if the transform is a bijection. Furthermore, when is a rigid rotation, translation, or combination thereof, the Jacobian determinant is always 1, and . If a random vector has mean zero and covariance matrix , with equality if and only if is jointly gaussian (see below). However, differential entropy does not have other desirable properties: It is not invariant under change of variables, and is therefore most useful with dimensionless variables. It can be negative. A modification of differential entropy that addresses these drawbacks is the relative information entropy, also known as the Kullback–Leibler divergence, which includes an invariant measure factor (see limiting density of discrete points). Maximization in the normal distribution Theorem With a normal distribution, differential entropy is maximized for a given variance. A Gaussian random variable has the largest entropy amongst all random variables of equal variance, or, alternatively, the maximum entropy distribution under constraints of mean and variance is the Gaussian. Proof Let be a Gaussian PDF with mean μ and variance and an arbitrary PDF with the same variance. Since differential entropy is translation invariant we can assume that has the same mean of as . Consider the Kullback–Leibler divergence between the two distributions Now