47 + 1.44 log O. (6.20) We thus find that the entropy of the normal or Gaussian distribution can be negative or zero if o < 1/2re, and increases as the logarithm of its variance or deviation. Short of any constraints on the variance O, the entropy is unbounded, or Hnormal too as o 80. We may compute the relative entropy, or KL distance, between any continuous PDF p(x) and the normal or Gaussian PDF, Inormal(x), according to Eq. (6.5) with mean and variance (xo, o2) and using natural logarithms: (6.21) where Hp(X) is the entropy of the source X with associated PDF p(x). We may apply the above definition to the case where p(x) is also a normal or Gaussian PDF with mean and variance (x1,02) and obtain (see Exercises): P D (x)] (6.22) As expected, the relative entropy or KL distance D vanishes if the two PDFs are identical, (xo, o2) = (x1,o2). Setting u = 02/02 and V = [(xo - X1)/oo]², we have D = (u2 - Inu-1-u)/2. It is easily checked that D reaches a minimum for = 1, Differential entropy namely, Dmin = v/2, which is zero for v=0. Also, D is infinite in the two lim- iting cases 0(01 0) or U +00(00 0), which correspond to p(x) 8(x - x1), q(x) 8(x - x0), respectively, where 8(x) is the Dirac or delta distri- bution. There are plenty of continuous-PDF types for which differential entropy comes out in closed-form or analytical expressions. A list of the most important PDFs comprises:6 Uniform, Normal or Gaussian, Exponential, Rayleigh, Beta, Cauchy, Chi-squared, Erlang, Gamma, Laplace, Logistic, Log-normal, Maxwell-Boltzmann, Generalized normal, Pareto, Student's t, Triangular, Weibull, Multivariate normal. Maximum entropy (continuous source) In Chapter 4, we addressed the problem of finding the maximum entropy. For discrete sources, this problem can be resolved with or without making restricting assumptions regarding the PDF, i.e., with or without assuming constraints. Without constraints, the straightforward result is that maximum entropy is reached with the uniform (discrete) PDF with N equiprobable events, i.e., p =