How is the variational lower bound for hard attention derived in Show, Attend and Tell
This is Jensen's inequality at work.
First of all, note that the first line can be rewritten as an expectation
$$\sum_{s} p(s \mid \mathbf{a}) \log p(\mathbf{y} \mid s, \mathbf{a}) = \mathbb{E}_{p(s|a)}[\log p(\mathbf{y} \mid s, \mathbf{a})]$$
Then Jensen's inequality gives (Note that a log function is a concave function so gives the opposite inequality to what is normally given when explaining Jensen's inequality with respect to convex functions):
$$\mathbb{E}_{p(s|a)}[\log p(\mathbf{y} \mid s, \mathbf{a})] \leq \log\mathbb{E}_{p(s|a)}[ p(\mathbf{y} \mid s, \mathbf{a})] $$
and then finally you can rewrite the Expectation as a summation.
$$\log\mathbb{E}_{p(s|a)}[ p(\mathbf{y} \mid s, \mathbf{a})] = \log \sum_{s} p(s \mid \mathbf{a}) p(\mathbf{y} \mid s, \mathbf{a})$$