Noting that σ has a major impact on the exploration-exploitation trade-off and can be either set to a prespecified value or parameterized by the policy network. When σ is pre-specified, the entropy regularization term in the PPO loss function becomes ineffective since σ is fixed and independently controlled. In the latter case where σ is being parametrized by the policy network, the entropy regularization term has a larger effect on exploration-exploitation.