class_docstr: Policy gradient objective, which maximizes the log-likelihood or likelihood-ratio scaled by the
  target reward value (specification key: `policy_gradient`).
  
  Args:
      importance_sampling (bool): Whether to use the importance sampling version of the policy
          gradient objective
          (<span style="color:#00C000"><b>default</b></span>: false).
      clipping_value (parameter, float > 0.0): Clipping threshold for the maximized value
          (<span style="color:#00C000"><b>default</b></span>: no clipping).
      early_reduce (bool): Whether to compute objective for aggregated likelihood instead of
          likelihood per action (<span style="color:#00C000"><b>default</b></span>: true).
      name (string): <span style="color:#0000C0"><b>internal use</b></span>.
      states_spec (specification): <span style="color:#0000C0"><b>internal use</b></span>.
      internals_spec (specification): <span style="color:#0000C0"><b>internal use</b></span>.
      auxiliaries_spec (specification): <span style="color:#0000C0"><b>internal use</b></span>.
      actions_spec (specification): <span style="color:#0000C0"><b>internal use</b></span>.
      reward_spec (specification): <span style="color:#0000C0"><b>internal use</b></span>.
code: def reference(
      self, *, states, horizons, internals, auxiliaries, actions, policy
  ):
      if self.early_reduce:
          log_probability = policy.log_probability(
              states=states,
              horizons=horizons,
              internals=internals,
              auxiliaries=auxiliaries,
              actions=actions,
          )
  
      else:
          log_probabilities = policy.log_probabilities(
              states=states,
              horizons=horizons,
              internals=internals,
              auxiliaries=auxiliaries,
              actions=actions,
          )
  
          def function(value, spec):
              return tf.reshape(tensor=value, shape=(-1, spec.size))
  
          log_probabilities = log_probabilities.fmap(
              function=function, zip_values=self.actions_spec
          )
          log_probability = tf.concat(
              values=tuple(log_probabilities.values()), axis=1
          )
  
      return log_probability
  
file_path: tensorforce/tensorforce/core/objectives/policy_gradient.py
imports: import numpy
  import tensorflow
  from tensorforce import util
  from tensorforce.core import parameter_modules, TensorSpec, tf_function, tf_util
  from tensorforce.core.objectives import Objective
label_desc: Use image node_id 4 for calling the PolicyGradient obj's underlying member method code with example usage: obj.reference() and returns: log_probability, tf