nd the upper bound of the redundancy, i.e., p < Pbound is always satisfied. The Shannon-Fano code redundancy bound is therefore Pbound = 1 bit/symbol, which means that in the general case, L** < H + 1, consistently with Eq. (8.15). In the specific case of dyadic sources (l* = -logm p is an integer for all x), we have L* = H, and p= 0 bit/symbol. We have seen from the preceding demonstration of Shannon's source-coding theorem, that there exists an optimum codeword-length assignment (1*(x), or more generally, l**(x)) for which the mean codeword length is minimized. What if we were to use a different assignment? Assume that this assignment takes the form i(x) = [-logm g(x)], where q(x) is an arbitrary distribution. It follows from this assignment that the mean codeword length becomes: 8.6 Shannon's source-coding theorem (8.16) Since <(8)x < it immediately follows from the above result that (8.17) <HM(X)+D(plq)+ 1. The result illustrates that a "wrong" choice of probability distribution results in a "penalty" of D(p)|q) for the smallest mean codeword length, which shifts the boundary interval according to L(X) E [HM(X) + D, HM(x) + D + 1]. Such a penalty can, however, be small, and a "wrong" probability distribution can, in fact, yield a mean codeword length L(X) smaller than L**(X). This fact will be proven in Chapter 9, which concerns optimal coding, and specifically Huffman codes. Since the above properties apply to any type of M-ary coding, we may wonder (inde- pendently of technological considerations for representing, acquiring, and displaying symbols) whether a specific choice of M could also lead to an absolute minimum for the expected length L(X). Could the binary system be optimal in this respect, considering the fact that the codewords have the longest possible lengths? A simple example with the English-character source (X ={A- Z}, 1982 survey, see Chapter 4) provides a first clue. Table 8.5 shows the mean codeword lengths *(X) for various multi-level codes: binary, ternary, quaternary, and 26-ary, with th