What's an entropy measure for the sequence $0, 1, 0, 1, 0, 1, ...$? I'm puzzled by the measure $$E(S) = -\sum p(i) \lg p(i)$$ because I'm considering a source emitting $0, 1, 0, 1, 0, ...$ and having the maximum entropy as given by $E(S)$.  Shouldn't $E(S)$ measure the quantity of information?  The sequence that just repeats $1$ after $0$ seems to me to convey very little information --- contained in this very phrase.

Where am I wrong in my understanding of entropy and of $E(S)$?
Let's just fill it in! Your generator has two possible outcomes, both with certain probabilities. For the every element of your sequence, you're certain about its outcome, thus:

\begin{align*}
E(S)&amp;=-p_0\log p_0 - p_1\log p_1\\
    &amp;=-p_0\log p_0 - (1-p_0)\log (1-p_0)\\
    &amp;\mathrel{\mathop{=}_{p_0\to1}} -1\log 1 - 0\\
    &amp;=-1\cdot0=0
\end{align*}

Seems like your source does not convey any information (zero bits), because indeed you already know what it will produce! Talking about entropy of a source only makes sense when the source has a stochastic behaviour; when it has an element of unpredictability.