wever, such a generalization is nontrivial, and far from being mathematically straightforward, as we shall see! A first argument that revealed the above issue was provided by Shannon in his seminal paper. 2 Our attention is first brought to the fact that, unlike discrete sums, integral sums are defined within a given coordinate system. Considering single integrals, such as in Eq. (6.1), and using the relation p(x)dx = p(y)dy, we have (6.7) =H(X)-C. Sxmin P(x)dx, meaning that the PDF p(x), if it exists, is positive and is integrable P(x y y) the interval The term "differential" comes from the fact that the probability is defined as P(x y y) = over considered, [Xmin,y], and is also the derivative of P(x < y) with respect to y, i.e., C.E. Shannon, A mathematical theory of communication. Bell Syst. Tech. J., 27 (1948), 79-423, 623-56. This paper can be freely downloaded from http://cm.bell-labs.com/cm/ms/what/shannonday/paper.html Differential entropy This result establishes that the entropy of a continuous source is defined within some arbitrary constant shift C, as appearing in the last RHS term. For instance, it is easy to show that the change of variable y=ax, (a being a positive constant) translates into the entropy shift H(Y)=H(X)+log a. The shift can be positive or negative, depending on whether a is greater or less than two (or than e, for the natural logarithm), respectively. A second observation is that differential entropy can be nonpositive (H(X) 0) unlike in the discrete case where entropy is always strictly positive (H(X) > 0).4 A straightforward illustration of this upsetting feature is provided by the continuous uni- form distribution over the real or finite interval x E [a, b]; (6.8) From the definition in Eq. (6.1), it is easily computed that the differential entropy is H(X) = log2(b - a). The entropy is zero or negative if b - a<2 (or b - a < e, for the natural logarithm). The difference between discrete and differential entropy becomes trickier if one attempts to connect the two definitions. This 