A2C Continuous for Pendulum-v0 working implementation, negation for loss and entropy calculation
Again, the entropy equation coded was this:  entropy = K.sum(0.5 * (K.log(2. * np.pi * sigma_sq) + 1.)) which looks different from what's given in the textbook photo above.


They are the same after simple algebraic manipulations.
The entropy of a single variable Gaussian distribution with pdf $p(x|\mu, \sigma)$ is
\begin{align}
  \mathcal{H}(p) &amp; = ln(\sqrt{2 \pi e \sigma^2}) \\
                 &amp; = 0.5 \cdot ln(2 \pi e \sigma^2) \\
                 &amp; = 0.5 \cdot (ln(2 \pi \sigma^2) + 1)
\end{align}
In policy gradient, we assume these $\sigma$s are isotropic. Thus the total entropy is the sum above $\mathcal{H}$s.


  Also, why is the loss a negation? actor_loss = -exp_v? Is it negated because it is gradient ascent rather than gradient descent of the objective function for a policy gradient?


Yes. In policy gradient you would like to maximize the likelihood log_pdf of getting higher episode returns advantages. At the same time we prefer a policy of high entropy. Deep learning frameworks usually only provide optimizer for minimizing the loss. Therefore you negate exp_v as loss.