) and Bayes's theorem, p(x, y) = p(y)x)p(x), we find the joint probabilities p(y1,x1) = (1 - E)q, p(y1,x2) = (1 - q)e, p(y2,x1) = Eq and p(y2,x2) = (1 1-8)(1-q). - - Replacing these results into Eq. (12.9), it is easily found that H(Y|X) = -logs-(1-e)log(1-e)=f(e), (12.15) and, hence, H(X,Y)=H(X)+H(YX) = (12.16) =f(q)+f(e), H(X;Y) = H(Y) - H(Y|X) = f(q+e-2eq)-f(e) (12.17) These results can be commented on as follows. First, we observe that the entropy of the output source H(Y) = f(q+8-2eq) - is different from that of the input source H(X) = f(q). Both reach the same maximum of H = 1 for q = 0.5. Furthermore, for any values of the noise parameter E, 3 the output source entropy satisfies H(Y)=fq+-2eq)zf(q)=H(X), = = (12.18) which is illustrated by the family of curves shown in Fig. 12.6, for E = 0 to E = 0.35 (noting that the same curves are obtained with E 1 E). As seen from the figure, uncertainty introduced by channel noise (E) increases the output source entropy H(Y), which was expected. In the cases q = 0 or q = 1, which correspond to a deterministic source X (p(x1) = 0 and p(x2) = 1, or p(x1) = 1 and p(x2) = 0), the output source entropy is nonzero, or H(Y) = f(e) = f(1-e). - The corresponding entropy can, thus, be seen as representing the exact measure of the channel's noise. It is inferred from the family of curves in Fig. 12.6 that in the limit E 0.5, the output source entropy The parameter E defines the amount of noise in the binary symmetrical communications channel through the conditional probabilities p(y)|xx). However, it should be noted that because of the symmetry of the transition matrix, the same amount of noise is associated with both E and 1 - E. Thus, noise increases for increasing values of E in the interval [0, 1/2], and decreases for increasing values of E in the interval [1/2, 1]. Channel entropy E = 0.35 Probability q Figure 12.6 Input H(X) and output H(Y) source entropies of binary symmetric channel, as functions of the probability q = p(x1) and channel noise parameter E. reaches the unif