Prove that the upper bound in the Noiseless-coding theorem is strict
Take the probability distribution that you consider: there are two symbols, one with probability $\delta$, the other with probability $1-\delta$.

Any minimum redundancy code for this distribution has average codeword length $1$. Therefore the redundancy is $1 - h(\delta)$. Since $h(1/2) = 1$, $\lim_{\delta\to0} h(\delta)=0$ and $h$ is continuous, for every $0 &lt; \gamma \leq 1$ you can find $\delta &gt; 0$ such that $h(\delta) = \gamma$. Choosing $\gamma = 1 - \epsilon$, we get a distribution whose redundancy is $1-\gamma = \epsilon$.

It is natural to "outlaw" such distributions by requiring all probabilities to be smaller than some $\kappa$. For small $\kappa$, we can no longer find distributions whose redundancy is close to 1. It is natural to conjecture that as $\kappa$ goes to zero, the maximum redundancy also goes to zero. However, this is not the case: for arbitrarily small $\kappa$, there are distributions whose redundancy is roughly 0.086. See Gallager's Variations on a theme by Huffman, which establishes that the limiting value of the maximum redundancy is (roughly) 0.086.