Let $H(p) = \sum\_i p\_i\log\frac{1}{p\_i}$ be the entropy of $p$
and $KL(p, q) = \sum\_i p\_i\log\frac{p\_i}{q\_i}$ be the KL divergence between $p$ and $q$. Does it hold that $H(p) \le H(q) + KL(p, q)$?

If this is not true, can we bound $H(p)$ using $H(q)$ and $KL(p, q)$ in certain form?

Edit 1: The motivation of this problem is this. Suppose that we are a bunch of data points as features (say $\{x\_1, \dots, x\_m\}$). And we have different distributions of labels over them. Say the first distribution is $q$. We use $q$ for training, and somehow we achieved Bayes optimal classifier. The loss of this classifier is $H(q)$.

Now say the second distribution is $p$. We know that the Bayesian optimal classifier over $p$ achieves loss $H(p)$.

Now I want to capture what is the difference between these two optimal classifiers over $p$ and $q$? There are two natural ways to capture this:

1. $H(p) - H(q)$. This simply measures the absolute performance difference (namely if Bayes optimal classifier over $q$ is doing well, would the Bayes optimal classifier over $p$, possibly different though, also doing well). This boils down to exactly measure the difference of entropy of $p$ and $q$.
2. $KL(p, q)$. This arises if we apply cross entropy to $p$ and $q$, $\ell(p, q) = H(p) + KL(p, q)$. Which is about what happens if we actually use $q$ to predict $p$. In this case $KL(p, q)$ captures the divergence.

I basically want to ask if these 2 are related.