he source. For example, if some source symbols Lmin=H (X) (4.56) Discrete channels with memory 4/9 n - HLX) (4.57) Let us now consider the problem of the reliable transmission of messages through a noisy communications channel. We therefore need the best decoding scheme to enable us to obtain the input A theorem which is complementary to Shannon's first theorem sequence after seeing the received symbols. Assuming the input and applies to a channel in which the noise is Gaussian is known as alphabet alphabet and channel matrix the Shannon-Hartley theorem, which states that the channel capac- p(y,lxi) as before, and consider the special case where a sequence ity of a white, bandlimited Gaussian channel is given by Equation of symbols, chosen at random according to a known distribution, 4.58, where B is the channel bandwidth, S the signal power, and N p(x), is transmitted through the channel. For this given distribution is the total noise within the channel bandwidth. p(x), we wish to construct a decision scheme that minimises the overall probability of error. Such a scheme is called an ideal ob- (4.58) server. C bits/s Consider a sequence x = chosen in accordance with the distribution p(x). The probability that the output sequence This theorem, although restricted to Gaussian channels, is of is produced is given by Equation 4.64, which may also be fundamental importance, since many channels are approximately Gaussian, and the results obtained by assuming a Gaussian channel written as Equation 4.65. often provide a lower bound on the performance of a system opera- ting over a non-Gaussian channel. The Shannon-Hartley theorem indicates that a noiseless Gaussian channel has an (4.64) infinite capacity. However, when noise is present in the channel, the capacity does not approach infinity as the bandwidth is increased, since the noise power increases as the bandwidth increases, and the channel capacity reaches a finite upper limit with increasing band- width if the signal power remains constant. This limit is easily (4