Inequality over the entropy of an integer-valued random variable I stumbled upon this inequality over a course on Information Theory :


  If $Z\in\mathbb{N}​$ with finite mean, then $H(Z) \leq E(Z)\times h(\frac 1 {E(Z)})​$ where $h$ is the binary entropy function


I have been trying to prove it using the log-sum inequality :


  $ \sum_i a_i \log \frac {a_i}{b_i} \geq (\sum_i a_i) \log \frac {\sum_i a_i}{\sum_i b_i} $ for $a_i, b_i$ non-negative numbers


Using, for example, $a_i = P(Z=i)$ and $b_i = iP(Z=i)$.
But I couldn't really get throught it. 

Part of my problem with this inequality is that I don't understand it : where does this use of the binary entropy function come from ? Even the relation between entropy and the mean seems unnatural to me.
Man, that's a nasty inequality.

Anyways, given a certain expectation $E[Z]=\mu$, it is known that the distribution that maximizes $H(Z)$ while keeping $E[Z]=\mu$ 
 is the the Geometric Distribution with parameter $p=1/\mu$. 

Then, it is easy to check that the inequality is satisfied (in fact the equality holds).
Indeed, if $Z\sim Geo(p)$ then $E[Z]=1/p$ and $H(Z)=-\log p -\frac{1-p}{p}\log(1-p)$.
Since this distribution maximizes $H(Z)$, any other distribution with the same expectation will have a lower left-hand side, while the right-hand side won't change (since $E[Z]=\mu$ didn't change). This proves the claim.