e best known model for a symmetric channel is the binary symmetric channel shown in Figure 4.2. As an example of the calculation of the channel capacity, a closed form expression for a Also as before, Equations 4.39 and 4.40 may be obtained. symmetric channel will be given. (4.39) 1 - p (4.40) From the expression for the average mutual information, it is seen therefore that the average mutual information depends both on the channel characteristics expressed in terms of the elements of the Input channel matrix, and also on the input probability distribution, which Output is clearly independent of the channel. Therefore, by changing the input probability distribution, the average mutual information will change, and we can define the 'channel capacity' in terms of the maximum average mutual information with respect to the input probability distribution. We can therefore write the channel capacity as in Equation 4.41. C = max I (XXY) (4.41) 1 - p The calculation of C is a constrained optimisation, since the constraints of Equations 4.42 and 4.43 have to apply to p(x). Figure 4.2 Binary symmetric channel p(x).=0 (4.42) Let us consider a symmetric channel with input alphabet X 1''..,XM' (4.43) and output alphabet and a channel matrix with row prob- Ep(x) = = abilities P1,...,PL and column probabilities Since H(Y)X) does not depend on the input distribution, the problem of 4/8 Information theory maximising the information reduces to the problem of maximising the output uncertainty H(Y) as in Equation 4.44. ((XY)-H(Y)-H(YX) = - (4.44) ((8,1-B) It is known that Equation 4.45 holds with equality if and only if all values of Y are equally likely. H(Y) : log L (4.45) Therefore, if an input distribution can be found for which all values of Y have the same probability, then that input distribution would maximise I (XY). The uniform distribution does this, since Equation 4.46 holds and for a uniform distribution Equation 4.47 holds. (4.46) (4.47) However, the term (,) is the sum of the entries in the jth Figure 4.3 Capacity of 