How should we interpret all the different metrics in reinforcement learning?
As you said, generally the most important one is reward per episode. If this isn't increasing overall, there's a problem (of course this metric can fluctuate, I mean to say that macroscopically it should increase).
Policy loss (I assume you mean the &quot;actor loss&quot;?) is generally harder to interpret. You should think of this more as a source of gradients and not necessarily a good indicator of how well your agent is performing.
I'm not really sure why you'd be monitoring the value during training. Value loss, however, is basically equivalent to value loss in value based methods like Q-learning, for example. So this one should be decreasing overall. Otherwise the baselines you compute to reduce variance in the policy gradient will either be less effective, or even harmful.
Entropy is a nice quantity to measure, because it's a good indicator of how much your agent is exploring. If you see that your agent is not achieving high returns and the entropy is really low, this means that your policy has converged to a suboptimal one. If the entropy is really high, this means the agent is acting fairly randomly (so it's basically exploring a lot). Ideally the entropy should decrease over time, so your policy becomes more deterministic (less exploration) as it reaches an optimum.