hannon's entropy is not an asymptotic limit. Rather, it is exactly defined for any source having a finite number of symbols N. We observe then from the definition that Shannon's entropy is the average value of the quantity I(x) = -logp(x), namely, H = (I) = - (log p), where I(x) is the information measure associated with a symbol x of probability p(x). The entropy of a source is, therefore, the average amount of information per source symbol, which we may also call the source information. If all symbols are equiprobable (p(x) = 1/N), the source entropy is given by H = = (4.7) which is equal to the information I = log N of all individual symbols. We will have several occasions in the future to use such a property. What is the unit of entropy? Were we to choose the natural logarithm, the unit of H would be nat/symbol. However, as seen in Chapter 3, it is more sensible to use the base-2 logarithm, which gives entropy the unit of bit/symbol. This choice is also consistent with the fact that N = 29 equiprobable symbols, i.e., with probability 1/N = 1/29, can be represented by log2 29 = q bits, meaning that all symbols from this source are made of exactly q bits. In this case, there is no difference between the source entropy, H = q, the symbol information, I = q, and the symbol bit length, l = q. In the rest of this book, it will be implicitly assumed that the logarithm is of base two. Let us illustrate next the source-entropy concept and its properties through practical examples based on various dice games and even more interestingly, on our language. Entropy in dice Here we will consider dice games, and see through different examples the relation between entropy and information. For a single die roll, the six outcomes are equiprob- able with probability p(x) = 1/6. As a straightforward application of the definition, or Eq. (4.7), the source entropy is thus (base-2 logarithm implicit): (4.8) We also have for the information: I = log 6 = 2.584 bits. This result means that it takes 3 bits (as the nearest upper integer)