(x,y)] = D[p(x)llq(x)] + D[p(y)x)||g(y|x)]. (6.14) In particular, it follows from Eqs. (6.13) and (6.10) that H(X)Y) < H(X) and H(Y|X) V H(Y), with equality if the sources are independent. Thus for continuous sources, conditioning reduces differential entropy, just as in the discrete case. I shall describe next a few examples of PDFs lending themselves to closed-form, or analytical definitions of differential entropy, with some illustrations regarding relative entropy and KL distance. Consider first the continuous uniform distribution, defined over the real interval of width : b - a, Eq. (6.8). As we have seen, the corresponding bit/symbol entropy is Huniform = log2(b-a). We note that the entropy Huniform is nonpositive if U < 1. The result shows that the entropy of a continuous uniform distribution of width increases as the logarithm of U. In the particular cases where u = 2N, with N being integer, then Huniform = N bit/symbol. In the limit u, N 8, or p(x) = 1/u = 2-N 0, corresponding to a uniform distribution of infinite width, the entropy is infinite, corresponding to an infinite number of degrees of freedom for source events having themselves an infinite information, I(x) = - log[p(x)] = N bits. We thus observe that, short of any constraints on the definition interval, or PDF mean, the entropy is unbounded, or Huniform +00 as N We may compute the relative entropy, or KL distance, between any continuous PDF p(x) defined over the domain X = [a, b] with u = (b - a) =2n and the corresponding uniform PDF, which we now call q(x) = 1/u=2-N, = according to Eq. (6.5): (6.15) Differential entropy = N Hp(X), where Hp(X) is the entropy of the source X and p(x) is the associated PDF. Since the KL distance is nonnegative, D(pllq) > 0, the above result implies that Hp(X) < N, meaning that N = log U represents the upper bound, or maximum entropy for all PDF defined over X. Consider next the continuous-exponential distribution, which is defined as p(x) = de-ix with x > 0 (nonnegative real), a = 1/(x) = 1/T a strictly positive