class_docstr: An ActorPolicy that also returns policy_info needed for PPO training.
  
  This policy requires two networks: the usual `actor_network` and the
  additional `value_network`. The value network can be executed with the
  `apply_value_network()` method.
  
  When the networks have state (RNNs, LSTMs) you must be careful to pass the
  state for the actor network to `action()` and the state of the value network
  to `apply_value_network()`. Use `get_initial_value_state()` to access
  the state of the value network.
code: def _apply_actor_network(
      self, time_step, policy_state, training=False
  ):
      observation = time_step.observation
      if self._observation_normalizer:
          observation = self._observation_normalizer.normalize(
              observation
          )
  
      return self._actor_network(
          observation,
          time_step.step_type,
          network_state=policy_state,
          training=training,
      )
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  from typing import Optional
  import gin
  import tensorflow
  from tf_agents.agents.ppo import ppo_utils
  from tf_agents.distributions import utils
  from tf_agents.networks import network
  from tf_agents.policies import actor_policy
  from tf_agents.specs import tensor_spec
  from tf_agents.trajectories import policy_step
  from tf_agents.trajectories import time_step
  from tf_agents.typing import types
  from tf_agents.utils import common
  from tf_agents.utils import nest_utils
  from tf_agents.utils import tensor_normalizer
label_desc: Use image node_id 4 for calling the PPOPolicy obj's underlying member method code with example usage: obj._apply_actor_network(time_step, policy_state, training) and returns: self