class_docstr: An ActorPolicy that also returns policy_info needed for PPO training.
  
  This policy requires two networks: the usual `actor_network` and the
  additional `value_network`. The value network can be executed with the
  `apply_value_network()` method.
  
  When the networks have state (RNNs, LSTMs) you must be careful to pass the
  state for the actor network to `action()` and the state of the value network
  to `apply_value_network()`. Use `get_initial_value_state()` to access
  the state of the value network.
docstr: Apply value network to time_step, potentially a sequence.
  
  If observation_normalizer is not None, applies observation normalization.
  
  Args:
    observations: A (possibly nested) observation tensor with outer_dims
      either (batch_size,) or (batch_size, time_index). If observations is a
      time series and network is RNN, will run RNN steps over time series.
    step_types: A (possibly nested) step_types tensor with same outer_dims as
      observations.
    value_state: Optional. Initial state for the value_network. If not
      provided the behavior depends on the value network itself.
    training: Whether the output value is going to be used for training.
  
  Returns:
    The output of value_net, which is a tuple of:
      - value_preds with same outer_dims as time_step
      - value_state at the end of the time series
docstr_tok: ["Apply","value","network","to","time_step",",","potentially","a","sequence",".","If","observation_normalizer","is","not","None",",","applies","observation","normalization",".","Args",":","observations",":","A","(","possibly","nested",")","observation","tensor","with","outer_dims","either","(","batch_size",",",")","or","(","batch_size",",","time_index",")",".","If","observations","is","a","time","series","and","network","is","RNN",",","will","run","RNN","steps","over","time","series",".","step_types",":","A","(","possibly","nested",")","step_types","tensor","with","same","outer_dims","as","observations",".","value_state",":","Optional",".","Initial","state","for","the","value_network",".","If","not","provided","the","behavior","depends","on","the","value","network","itself",".","training",":","Whether","the","output","value","is","going","to","be","used","for","training",".","Returns",":","The","output","of","value_net",",","which","is","a","tuple","of",":","-","value_preds","with","same","outer_dims","as","time_step","-","value_state","at","the","end","of","the","time","series"]
code: def apply_value_network(
      self,
      observations: types.NestedTensor,
      step_types: types.Tensor,
      value_state: Optional[types.NestedTensor] = None,
      training: bool = False,
  ) -> types.NestedTensor:
      """Apply value network to time_step, potentially a sequence.
  
      If observation_normalizer is not None, applies observation normalization.
  
      Args:
        observations: A (possibly nested) observation tensor with outer_dims
          either (batch_size,) or (batch_size, time_index). If observations is a
          time series and network is RNN, will run RNN steps over time series.
        step_types: A (possibly nested) step_types tensor with same outer_dims as
          observations.
        value_state: Optional. Initial state for the value_network. If not
          provided the behavior depends on the value network itself.
        training: Whether the output value is going to be used for training.
  
      Returns:
        The output of value_net, which is a tuple of:
          - value_preds with same outer_dims as time_step
          - value_state at the end of the time series
      """
      if self._observation_normalizer:
          observations = self._observation_normalizer.normalize(
              observations
          )
      return self._value_network(
          observations, step_types, value_state, training=training
      )
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  from typing import Optional
  import gin
  import tensorflow
  from tf_agents.agents.ppo import ppo_utils
  from tf_agents.distributions import utils
  from tf_agents.networks import network
  from tf_agents.policies import actor_policy
  from tf_agents.specs import tensor_spec
  from tf_agents.trajectories import policy_step
  from tf_agents.trajectories import time_step
  from tf_agents.typing import types
  from tf_agents.utils import common
  from tf_agents.utils import nest_utils
  from tf_agents.utils import tensor_normalizer
label_desc: Use image node_id 3 for calling the PPOPolicy obj's underlying member method code with example usage: obj.apply_value_network(observations, step_types, value_state, training) and returns: self