the roughest possible condition of typicality. 12 To recall, the first theorem from Shannon, the source-coding theorem, was described in Chapter 8, see Eq. (8.15). Channel capacity and coding theorem CCT can be stated in a number of different and equivalent ways. A possible definition, which reflects the original one from Shannon, 13 is: Given a noisy communication channel with capacity C, and a symbol source X with entropy H(X), which is used at an information rate R < C, there exists a code for which message symbols can be transmitted through the channel with an arbitrary small error E. The demonstration of the CCT rests upon the subtle notion of "typical sets," as analyzed in the previous section. It proceeds according to the following steps: Assume the input source messages X to have a length (number of symbols) N. With independent symbol outcomes, the corresponding extended-source entropy is H(X") = nH(X). The typical set of Xn roughly contains 2nH(X) possible sequences, which represent the most probable input messages. Call y the output message sequences received after transmission through the noisy channel. The set of y sequences corresponds to a random source Yn of entropy H(Y") = n(((H(Y). The channel capacity C is the maximum of the mutual information H(X; Y) = H(X) - H(X)Y). Assume that the source X corresponds (or nearly corresponds) to this optimal condition. Refer now to Fig. 13.5 and observe that: The typical set of Yn roughly contains 2nH(Y) possible sequences, which represent the most probable output sequences y, other outputs having a comparatively small total probability. Given an output sequence Yj, there exist 2nH(X|Y) most likely and equiprobable input sequences x (also called "reasonable causes"), other inputs having comparatively small total probability. Given an input sequence Xi, there exist 2nH(Y|X) most likely and equiprobable output sequences y (also called "reasonable effects"), other outputs having comparatively small total probability. Assume next that the originator is using the c