class_docstr: An ActorPolicy that also returns policy_info needed for PPO training.
  
  This policy requires two networks: the usual `actor_network` and the
  additional `value_network`. The value network can be executed with the
  `apply_value_network()` method.
  
  When the networks have state (RNNs, LSTMs) you must be careful to pass the
  state for the actor network to `action()` and the state of the value network
  to `apply_value_network()`. Use `get_initial_value_state()` to access
  the state of the value network.
code: def _distribution(self, time_step, policy_state, training=False):
      if not policy_state:
          policy_state = {
              "actor_network_state": (),
              "value_network_state": (),
          }
      else:
          policy_state = policy_state.copy()
  
      if "actor_network_state" not in policy_state:
          policy_state["actor_network_state"] = ()
      if "value_network_state" not in policy_state:
          policy_state["value_network_state"] = ()
  
      new_policy_state = {
          "actor_network_state": (),
          "value_network_state": (),
      }
  
      (
          distributions,
          new_policy_state["actor_network_state"],
      ) = self._apply_actor_network(
          time_step,
          policy_state["actor_network_state"],
          training=training,
      )
  
      if self._collect:
          policy_info = {
              "dist_params": ppo_utils.get_distribution_params(
                  distributions,
                  legacy_distribution_network=isinstance(
                      self._actor_network, network.DistributionNetwork
                  ),
              )
          }
  
          if not self._compute_value_and_advantage_in_train:
              # If value_prediction is not computed in agent.train it needs to be
              # computed and saved here.
              (
                  policy_info["value_prediction"],
                  new_policy_state["value_network_state"],
              ) = self.apply_value_network(
                  time_step.observation,
                  time_step.step_type,
                  value_state=policy_state["value_network_state"],
                  training=False,
              )
      else:
          policy_info = ()
  
      # Disable lint for TF arrays.
      if not common.safe_has_state(
          new_policy_state["actor_network_state"]
      ) and not common.safe_has_state(
          new_policy_state["value_network_state"]
      ):
          new_policy_state = ()
      elif not common.safe_has_state(
          new_policy_state["value_network_state"]
      ):
          del new_policy_state["value_network_state"]
      elif not common.safe_has_state(
          new_policy_state["actor_network_state"]
      ):
          del new_policy_state["actor_network_state"]
  
      return policy_step.PolicyStep(
          distributions, new_policy_state, policy_info
      )
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  from typing import Optional
  import gin
  import tensorflow
  from tf_agents.agents.ppo import ppo_utils
  from tf_agents.distributions import utils
  from tf_agents.networks import network
  from tf_agents.policies import actor_policy
  from tf_agents.specs import tensor_spec
  from tf_agents.trajectories import policy_step
  from tf_agents.trajectories import time_step
  from tf_agents.typing import types
  from tf_agents.utils import common
  from tf_agents.utils import nest_utils
  from tf_agents.utils import tensor_normalizer
label_desc: Use image node_id 6 for calling the PPOPolicy obj's underlying member method code with example usage: obj._distribution(time_step, policy_state, training) and returns: policy_step