as the event that message i is jointly typical with the sequence received when message 1 is sent. We can observe that as goes to infinity, if for the channel, the probability of error will go to 0. Finally, given that the average codebook is shown to be "good," we know that there exists a codebook whose performance is better than the average, and so satisfies our need for arbitrarily low error probability communicating across the noisy channel. Weak converse for discrete memoryless channels Suppose a code of codewords. Let W be drawn uniformly over this set as an index. Let and be the transmitted codewords and received codewords, respectively. using identities involving entropy and mutual information since X is a function of W by the use of Fano's Inequality by the fact that capacity is maximized mutual information. The result of these steps is that . As the block length goes to infinity, we obtain is bounded away from 0 if R is greater than C - we can get arbitrarily low rates of error only if R is less than C. Strong converse for discrete memoryless channels A strong converse theorem, proven by Wolfowitz in 1957, states that, for some finite positive constant . While the weak converse states that the error probability is bounded away from zero as goes to infinity, the strong converse states that the error goes to 1. Thus, is a sharp threshold between perfectly reliable and completely unreliable communication. Channel coding theorem for non-stationary memoryless channels We assume that the channel is memoryless, but its transition probabilities change with time, in a fashion known at the transmitter as well as the receiver. Then the channel capacity is given by The maximum is attained at the capacity achieving distributions for each respective channel. That is, where is the capacity of the ith channel. Outline of the proof The proof runs through in almost the same way as that of channel coding theorem. Achievability follows from random coding with each symbol chosen randomly from the capacity achieving distribution for that particular channel. Typicality arguments use the definition of typical sets for non-stationary sources defined in the asymptotic equipartition property article. The technicality of lim inf comes into play when does not converge. See also Asymptotic equipartition property (AEP) Fano's inequality Rate–distortion theory Shannon's source coding theorem Shannon–Hartley theorem Turbo code Notes References Cover T. M.,