 large k and n an encoding scheme that maps source words Eb/No 2R for reliable transmission. Here Eb/No is of length k into channel words of length n and a decoder called the bit energy to noise spectral density ratio and that maps channel output words of length n into repro- 22R-1 2R is the Shannon bound. Since R > 0, we must also duced words of length k such that the expected distortion is have Eb/No > loge 2 or 10 log10 Eb/No > -1.6 dB, which at most D and the transmission rate k/n is at least r. is called the theoretical Shannon limit. The encoding scheme promised by the theorem is In the time-continuous AWGN channel, the relation- a concatenation of a lossy compression scheme and ship between the transmitted signal s(t) (the output of a channel encoding scheme. The theorem is also known the modulator) and the received signal r(t) (the input of as the separation theorem because the lossy compression the demodulator) is r(t) = s(t) + n(t), where n(t) is scheme and the channel encoding scheme can be a white Gaussian noise. The capacity of a band-limited designed independently. AWGN channel is [15] 3.4.2.4 Extensions C = W1082(1+Now) = bits per second, Shannon's theorems can be extended to more general information sources. For example, we say that a discrete where W is the channel bandwidth in Hz, No/2 is the source Xn is stationary if the random process Xn is power spectral density of the noise, and P is a constraint stationary. The nth marginal entropy of a stationary on the average power. When P is much smaller than source is Hn = H(Xn|Xn-1,....X1) One can show that NoW, the channel is called a wideband AWGN channel. when the source is stationary, the sequence Hn is de- One can prove [11] that if binary modulation is used and creasing and bounded below by zero. This allows us to the demodulated signal is sampled at a rate of 2W, then define the entropy of a stationary source as follows. Eb/No must be larger than the practical Shannon limit of 0.2 dB to achieve a BER of 10 for a rate of Definition 3.4.5 (Entro