class_docstr: :param policy_gradient_rescaler: (PolicyGradientRescaler)
      The value that will be used to rescale the policy gradient
  
  :param apply_gradients_every_x_episodes: (int)
      The number of episodes to wait before applying the accumulated gradients to the network.
      The training iterations only accumulate gradients without actually applying them.
  
  :param beta_entropy: (float)
      The weight that will be given to the entropy regularization which is used in order to improve exploration.
  
  :param num_steps_between_gradient_updates: (int)
      Every num_steps_between_gradient_updates transitions will be considered as a single batch and use for
      accumulating gradients. This is also the number of steps used for bootstrapping according to the n-step formulation.
  
  :param gae_lambda: (float)
      If the policy gradient rescaler was defined as PolicyGradientRescaler.GAE, the generalized advantage estimation
      scheme will be used, in which case the lambda value controls the decay for the different n-step lengths.
  
  :param estimate_state_value_using_gae: (bool)
      If set to True, the state value targets for the V head will be estimated using the GAE scheme.
code: def __init__(self):
      super().__init__()
      self.policy_gradient_rescaler = PolicyGradientRescaler.A_VALUE
      self.apply_gradients_every_x_episodes = 5
      self.beta_entropy = 0
      self.num_steps_between_gradient_updates = (
          5000  # this is called t_max in all the papers
      )
      self.gae_lambda = 0.96
      self.estimate_state_value_using_gae = False
  
imports: from typing import Union
  import numpy
  import scipy.signal
  from rl_coach.agents.policy_optimization_agent import PolicyOptimizationAgent, PolicyGradientRescaler
  from rl_coach.architectures.embedder_parameters import InputEmbedderParameters
  from rl_coach.architectures.head_parameters import PolicyHeadParameters, VHeadParameters
  from rl_coach.architectures.middleware_parameters import FCMiddlewareParameters
  from rl_coach.base_parameters import AlgorithmParameters, NetworkParameters, AgentParameters
  from rl_coach.exploration_policies.categorical import CategoricalParameters
  from rl_coach.exploration_policies.continuous_entropy import ContinuousEntropyParameters
  from rl_coach.logger import screen
  from rl_coach.memories.episodic.single_episode_buffer import SingleEpisodeBufferParameters
  from rl_coach.spaces import DiscreteActionSpace, BoxActionSpace
  from rl_coach.utils import last_sample
label_desc: Use image node_id 1 to create a new ActorCriticAlgorithmParameters object from inherited base classes: AlgorithmParameters with example: obj = ActorCriticAlgorithmParameters()