 not possible if the infor- mation processed is greater than the channel capacity. However, in general, the calculation of the channel capacity is a difficult prob- This is a very important equation, since if we are given the input a priori probabilities p(x) and the channel matrix, then we may Before calculating the channel capacity for some simple channel obtain the probabilities of the various output symbols from the models, it is useful to introduce certain classes of channel, as above equation. follows: 1. A channel is lossless if H (X Y) = 0 for all input distribution- 4.5.1 Channel capacity S, which means that the input is determined by the output, and hence no transmission errors can occur. Regardless of whatever means, or channel, is used for transmission, 2. A channel is deterministic if p(y)|xi) = 1 or 0 for all i, j, there is a maximum rate of transmission, called the 'capacity' of the which means that Y is determined by X, and hence channel, which is determined by the intrinsic properties of the H(Y)X) = 0 for all input distributions. channel and is independent of the content of the transmitted infor- 3. A channel is noiseless if it is lossless and deterministic. mation and the way it is encoded. This is measured in bits per 4. A channel is useless or zero capacity if I (XY) = 0 for all second. As an example, in order to transmit a colour television input distributions. picture, a channel with a capacity of about 200 million bits per 5. A channel is symmetric if each row of the transition matrix second is required. contains the same set of numbers, and if each column contains For discrete memoryless channel with transition probabilities the same set of numbers (different in general for the row set). p(yk/xj) as before, the average mutual information between the output and the input is given by Equation 4.38. It is a consequence of the definition of the symmetric channel that H(Y)) is independent of the input distribution p(x), and depends p(yklxj) (4.38) only on the channel probabilities p(y,lxi). Th