PPO, A2C for continuous action spaces, math and code
Packt Publishing's "Deep Reinforcement Learning Hands-On" has an entire chapter on continuous action spaces.  Here is the math in the book:



and the code accompanying the book:  code repo for book

The code for the math above is in:


  The complete source code is in Chapter14/02_train_a2c.py ,
  Chapter14/lib/model.py and Chapter14/lib/common.py . Most of the code
  will already be familiar to you, so the following includes only the
  parts that differ. Let's start with the model class defined in
  Chapter14/lib/model.py .