Here is the reformatted version for clarity as requested:

Deep neural networks have been implemented to represent actor and critic functions for non-trivial observation and action spaces.

The actor network maps each possible observation in the agent's observation space to a probability distribution over possible actions in the action space, while the critic network maps each possible observation to a corresponding expected value estimate.

In practice, some of the early layers are often shared between the actor and critic networks, as they both need to do some of the same processing, such as learning edge-detection in a pixel-valued observation space.
spam fails_task lang_mismatch pii not_appropriate hate_speech sexual_content quality toxicity humor helpfulness creativity violence