 constant, and T = (x) the PDF mean, also called the 1/e lifetime (Chapter 2). An elementary calcu- lation (see Exercises) while taking the natural logarithm in the entropy definition yields the nat/symbol entropy: (6.16) Thus, the entropy of a continuous-exponential distribution Hexp can be negative or null if T < 1/e and increases as the logarithm of the lifetime T. Short of any constraints on the mean or lifetime, the entropy is unbounded, or Hexp Assuming T = eN-1, where N is an integer, we obtain Hexp = N nat/symbol, which compares with the entropy of the uniform distribution, Huniform = N bit/symbol, for a source defined over any real interval x E [a, b] having the width = (b-a) - = 2N. We may compute the relative entropy, or KL distance, between any continuous PDF p(x) defined over the domain x > 0 and the exponential PDF, which we now call q(x) = de-ix, according to Eq. (6.5), and using natural logarithms: (6.17) / In(A) =1(x),-In(A)-H,(X), where Hp(X) is the entropy of X with PDF p(x) and (x), p is the PDF mean value. From this result, we can determine the relative entropy or KL distance between two exponential PDFs, i.e., Pexp(x) = -() and 9exp(x) = de-ax. which gives (6.18) 6.1 Entropy of continuous sources Setting = N/U, we have D = U - In(eu). It can easily be checked that D reaches a minimum of zero for = 1 (A = M, or p(x) = q(x)), as expected. In the limits a (p(x) (x)) or u (p(x) -> (x)), the KL distance D is infinite, but (x), which emulates a step or uniform distribution over the interval x E [0, +00], is not a valid PDF. It is left as an exercise to study an exponential PDF variant that is defined over a finite interval x E [0, m], and to reconcile the infinite limit with the uniform-distribution case. Consider next the case of the normal or Gaussian distribution, p(x), as defined in Eq. (2.21). Using first natural logarithms, the differential entropy calculation yields: (6.19) where we used the property <(x - x0)2 = (x2) (x2) o2. In bit/symbol units, this result gives 2.047 + 0.72 log o2 = 2.0