a binary symmetric channel column of the channel matrix, and since the channel is symmetric, Equation 4.48 is obtained, which is independent of j, and hence P(y) does not depend on j, or equivalently all values of Y have the same are known to be more probable than others, then this feature may be probability. exploited by assigning short code words to frequent source symbols and vice versa, e.g. the Morse code. (4.48) The average length of a code word, I can be written as in Equation 4.53. (4.53) The maximum, given by Equation 4.49, is therefore attainable and the channel capacity can be written as in Equation 4.50 since Equation 4.51 is also true. This represents the average number of bits per source symbol used H(Y) = log L (4.49) in the source encoding process. If Lmin represents the minimum possible value of L, the coding efficiency of the source encoder is (4.50) defined as in Equation 4.54 and the source encoder is said to be efficient when n approaches unity. (4.51) (4.54) One question of importance is how to determine the minimum length code. The answer to this is given in Shannon's first theorem, For the binary symmetric channel shown in Figure 4.2, the chan- also called the source coding theorem. This states that given a nel capacity can be written as in Equation 4.52 and the variation discrete memoryless source of entropy H(X), then the average code with B is shown in Figure 4.3. word length L for any source encoding is bounded as in Equation 4.55. =1-H(6,1-6) (4.52) (4.55) Therefore, the entropy H(X) represents a fundamental limit on the 4.5.2 Source encoding average number of bits per source symbol necessary to represent a discrete memoryless source. Thus Equation 4.56 can be obtained An important problem in communications is how to efficiently and the efficiency of the source encoder may be written in terms of represent data generated by a discrete source. This is called source the entropy as in Equation 4.57. encoding. For a source encoder to be efficient, one needs knowledge of the statistics of t