class_docstr: :param num_steps_between_copying_online_weights_to_target: (StepMethod)
      The number of steps between copying the online network weights to the target network weights.
  
  :param apply_gradients_every_x_episodes: (int)
      The number of episodes between applying the accumulated gradients to the network. After every
      num_steps_between_gradient_updates steps, the agent will calculate the gradients for the collected data,
      it will then accumulate it in internal accumulators, and will only apply them to the network once in every
      apply_gradients_every_x_episodes episodes.
  
  :param num_steps_between_gradient_updates: (int)
      The number of steps between calculating gradients for the collected data. In the A3C paper, this parameter is
      called t_max. Since this algorithm is on-policy, only the steps collected between each two gradient calculations
      are used in the batch.
  
  :param targets_horizon: (str)
      Should be either 'N-Step' or '1-Step', and defines the length for which to bootstrap the network values over.
      Essentially, 1-Step follows the regular 1 step bootstrapping Q learning update. For more information,
      please refer to the original paper (https://arxiv.org/abs/1602.01783)
code: def __init__(self):
      super().__init__()
      self.num_steps_between_copying_online_weights_to_target = (
          EnvironmentSteps(10000)
      )
      self.apply_gradients_every_x_episodes = 1
      self.num_steps_between_gradient_updates = (
          5  # this is called t_max in all the papers
      )
      self.targets_horizon = "N-Step"
  
imports: from typing import Union
  import numpy
  from rl_coach.agents.policy_optimization_agent import PolicyOptimizationAgent
  from rl_coach.agents.value_optimization_agent import ValueOptimizationAgent
  from rl_coach.architectures.embedder_parameters import InputEmbedderParameters
  from rl_coach.architectures.head_parameters import QHeadParameters
  from rl_coach.architectures.middleware_parameters import FCMiddlewareParameters
  from rl_coach.base_parameters import AlgorithmParameters, AgentParameters, NetworkParameters
  from rl_coach.core_types import EnvironmentSteps
  from rl_coach.exploration_policies.e_greedy import EGreedyParameters
  from rl_coach.memories.episodic.single_episode_buffer import SingleEpisodeBufferParameters
  from rl_coach.utils import last_sample
label_desc: Use image node_id 1 to create a new NStepQAlgorithmParameters object from inherited base classes: AlgorithmParameters with example: obj = NStepQAlgorithmParameters()