P(Y) = [p(y1), p(D), p(y2)] is defined as follows: (12.6) Channel entropy (1-e 0 p(y,x)=== Originator Recipient source p(12)x2)=1-8 source Noisy channel Figure 12.3 Binary erasure channel, showing the originator source X={x1,x2}, the recipient source Y= {y1,y2}, the conditional probabilities p(yj|xi), and the corresponding transition matrix P(Y|X), with the recipient source. Example 12.3: Noisy typewriter This corresponds to a "typewriter" version of the binary symmetric channel, called the noisy typewriter, which is illustrated in Fig. 12.4. As seen from the figure, the source symbols are the characters {A, B, ,C,...ZZ, arranged in a linear keyboard sequence. When a given symbol character is input to the keyboard, there are equal chances of outputting the same character or one of its two keyboard neighbors. For instance, letter B as the input yields either A, B, or C as the output, with p(A|B) = p(B I B) = p(C|B) = 1/3, and SO on for the 26 letters (input A yielding Z, A, or B), as illustrated in the figure. The corresponding transition matrix (reduced here for clarity to six input/output symbol characters, is the following: 111000 (12.7) 000111 100011 Example 12.4: Asymmetric channel with nonoverlapping outputs This corresponds to an asymmetric channel with nonoverlapping outputs, which maps source X = {x1,x2} to source Y = {y1, Y2, Y3, Y4} according to the transition proba- bilities defined in Fig. 12.5. The property of this channel is that each of the outputs 12.2 Nonbinary and asymmetric discrete channels p(yx) = 1/3 Originator Recipient source source Figure 12.4 Noisy typewriter, showing the originator source X = {x1,x2}, the recipient source Y = {y1, ,y2}, the conditional probabilities p(yj|xi), and the corresponding transition matrix P(Y|X), with X,Y={A, B, C,...,Z}. p(xx)=0 Originator Recipient source source Noisy channel Figure 12.5 Asymmetric channel with nonoverlapping outputs, showing the originator source X = {x1,x2}, the recipient source Y ={y1,y2}, the conditional probabilities p(y)|xi), and the c