docstr: Loss for PPO Value network.
  Schulman implemented this extension in OpenAI baselines for PPO2
  See https://github.com/openai/baselines/blob/master/baselines/ppo2/ppo2.py#L72
  
  :param clip_likelihood_ratio_using_epsilon: epsilon to use for likelihood ratio clipping.
  :param weight: scalar used to adjust relative weight of loss (if using this loss with others).
  :param batch_axis: axis used for mini-batch (default is 0) and excluded from loss aggregation.
docstr_tok: ["Loss","for","PPO","Value","network",".","Schulman","implemented","this","extension","in","OpenAI","baselines","for","PPO2","See","https",":","\/\/github.com\/openai\/baselines\/blob\/master\/baselines\/ppo2\/ppo2.py","#","L72",":","param","clip_likelihood_ratio_using_epsilon",":","epsilon","to","use","for","likelihood","ratio","clipping",".",":","param","weight",":","scalar","used","to","adjust","relative","weight","of","loss","(","if","using","this","loss","with","others",")",".",":","param","batch_axis",":","axis","used","for","mini-batch","(","default","is","0",")","and","excluded","from","loss","aggregation","."]
code: def __init__(
      self,
      clip_likelihood_ratio_using_epsilon: float,
      weight: float = 1,
      batch_axis: int = 0,
  ) -> None:
      """
      Loss for PPO Value network.
      Schulman implemented this extension in OpenAI baselines for PPO2
      See https://github.com/openai/baselines/blob/master/baselines/ppo2/ppo2.py#L72
  
      :param clip_likelihood_ratio_using_epsilon: epsilon to use for likelihood ratio clipping.
      :param weight: scalar used to adjust relative weight of loss (if using this loss with others).
      :param batch_axis: axis used for mini-batch (default is 0) and excluded from loss aggregation.
      """
      super(PPOVHeadLoss, self).__init__(
          weight=weight, batch_axis=batch_axis
      )
      self.weight = weight
      self.clip_likelihood_ratio_using_epsilon = (
          clip_likelihood_ratio_using_epsilon
      )
  
file_path: coach/rl_coach/architectures/mxnet_components/heads/ppo_v_head.py
imports: from typing import List, Tuple, Union
  from types import ModuleType
  import mxnet
  from mxnet.gluon import nn
  from rl_coach.architectures.mxnet_components.heads.head import Head, HeadLoss, LossInputSchema, NormalizedRSSInitializer
  from rl_coach.architectures.mxnet_components.heads.head import LOSS_OUT_TYPE_LOSS
  from rl_coach.base_parameters import AgentParameters
  from rl_coach.core_types import ActionProbabilities
  from rl_coach.spaces import SpacesDefinition
label_desc: Use image node_id 1 to create a new PPOVHeadLoss object from inherited base classes: HeadLoss with example: obj = PPOVHeadLoss(clip_likelihood_ratio_using_epsilon, weight, batch_axis)