orresponding transition matrix P(Y|X), with X = {x1,x2} and Y = {y1, Y2, Y3, Y4}. Channel entropy {y1, y2, y3, y4} has only one corresponding input {x1, x2}. While the output symbols are random, the input symbols are deterministic, which represents a special case of noisy channel where uncertainty does not increase, as discussed later in this chapter. It is easily established that the corresponding transition matrix is: (12.8) The four above examples of asymmetric, nonbinary channels will be used again in Chapter 13 to illustrate the concept of channel capacity. Channel entropy and mutual information With the examples of the previous section, we can now make a practical use of the abstract definitions introduced in Chapter 5 concerning conditional entropy (also called equiv- location), H(Y|X), joint entropy, H(X, Y), and mutual information, H(Y; X), between two sources X, Y. It is useful to recall here the corresponding definitions and their relations: H(Y|X) (12.9) H(X, Y) = H(Y, X) (12.10) H(X) + H(Y|X) H(Y) + H(X|Y), H(X; Y) = H(Y) - H(Y|X) = :H(X)-H(XY) - (12.11) = H(X)+H(Y)-H(X,Y). We consider first the elementary case of the symmetric binary channel, which was previously analyzed and illustrated in Fig. 12.1. The input probability distribution is, thus, defined by p(x1) = q and p(x2)=1 = - q, where q is a nonnegative real num- ber such that 0    1. According to definition, the entropy of the input source is H(X) = - - log p(x2) = f(q), where f(q) =-qlogq-(1-q)log(1-q) = (12.12) The function f(q), which was first introduced in Chapter 4, is plotted in Fig. 4.7. It is seen from the figure that it has a maximum of f(q) = 1 for q = 0.5, which corresponds to the case of maximal uncertainty in the source X. The output probabilities are calculated 12.3 Channel entropy and mutual information as follows: p(y1) =p(y1lx1)p(x1)+p(y1/x2)p(x2) = :(1-8)q+8(1-q) = 9+8-2Eq (12.13) p(y2) = 1 We find, thus, the entropy of the output source: H(Y)=f(q+-2eq). - (12.14) Using next the transition matrix P(Y|X) defined in Eq. (12.4