ix, we find how the two entropy definitions in the discrete-source and continuous-source cases connect. For the discrete case, we have, by definition: and for the continuous case: where the discrete and continuous distributions, which relate to the sources X (discrete) and X' (continuous) are called p(xi) and p'(x), respectively. Considering the continuous function p(x), we can decompose its integration domain into small bins of width A, which we label with the index j. The variable x belongs to the bin if the condition j A (j + 1)A is satisfied. The width A is chosen small enough SO that in any bin j there exists a value Xj, for which (j+1)^ p'(x)dx = p'(x))A (note that there is no functional relation between the discrete distribution p(xi) and the continuous distribution p'(xj) at points Xj of the integration domain). Equation (E3), however, defines a discrete distribution, which we shall call p"(xj) = p'(xj) A. Such a distribution carries over the discrete set X" = {xj}, for which Eq. (E3) is satisfied. According to the additivity property of the integrals, this distribution satisfies: (we note that the left-hand side of this equation is called the Riemann integral of the continuous function p'(x)). We have, thus, obtained a strict equivalence between the continuous distribution p'(x) and a discrete distribution p"(xj). We, thus, expect that See also: T.M. Cover and J.A. Thomas, Elements of Information Theory (New York: John Wiley & Sons, 1991). D. Feldmann, A Brief Tutorial On: Information Theory, Excess Entropy and Statistical Complexity (2002), available online at http://hornacek.coa.edu/dave/Tutorial/index.html Appendix E their respective entropies become very nearly equal if the bin interval A is chosen sufficiently small. The surprise is that it this not all the case, as I show next. The entropy H(X") of the discrete distribution p" related to the source X" is defined according to Eq. (E1): Substituting in Eq. (E5) the definition p"(xj) = p'(xj)A, and using the property in Eq. (E4), we obtain: H(X") = xj