class_docstr: Base trainer class for the DQN algorihtm variants.
  
  A base class for training variants of the DQN algorithm.
  Provides methods for q-values calculation, reward boosting, and also
  initializing and training counterfactual policy evaluation
  metrics estimators (see https://arxiv.org/abs/1811.00260 for detail).
  
  Attributes:
      rl_parameters: an object of a reagent.core.parameters.RLParameters
          data class.
      metrics_to_score: a list of strings representing reward metrics.
      evaluation_parameters: an object of a
          reagent.core.parameters.EvaluationParameters data class.
      q_network_loss: a loss function object, currently either
          torch.nn.functional.mse_loss or torch.nn.functional.smooth_l1_loss.
      reward_boosts: a torch tensor containing reward boost values at
          indices corresponding to respective discrete actions.
      calc_cpe_in_training: a boolean flag whether to calculate CPE metrics
          during training.
      reward_network: a network object mapping states to rewards and
          extra reward metrics. Given the number of discrete actions n and
          e.g. a set of k reward metrics `m_1`, `m_2`, ..., and `m_k`, the
          reward_network output has shape (batch_size, n * k), where
          the [:, (k - 1) * n : k * n] sub-tensor contains the metric `m_k`
          q-values estimates for each of the n actions. The order of metrics
          in the output is the following: the environment reward, followed by
          extra metrics sorted by their names in RewardOptions.metric_reward_values
          config.
      reward_network_optimizer: an optimizer object for training
          reward network.
      q_network_cpe: a network object mapping states to CPE metrics.
      q_network_cpe_target: a copy of q_network_cpe for training stability.
      q_network_cpe_optimizer: an optimizer object for training q_network_cpe.
      reward_idx_offsets: a flat torch tensor containing integer offsets of
          different reward metrics in the reward_network output.
      evaluator: an object of a reagent.evaluation.evaluator.Evaluator class.
docstr: Args:
      rl_parameters: an object of a reagent.core.parameters.RLParameters
          data class.
      metrics_to_score: a list of strings representing extra reward
          metrics, excluding the reward itself.
      actions: a list of string action names for available actions.
      evaluation_parameters: an object of a
          reagent.core.parameters.EvaluationParameters data class.
docstr_tok: ["Args",":","rl_parameters",":","an","object","of","a","reagent.core.parameters.RLParameters","data","class",".","metrics_to_score",":","a","list","of","strings","representing","extra","reward","metrics",",","excluding","the","reward","itself",".","actions",":","a","list","of","string","action","names","for","available","actions",".","evaluation_parameters",":","an","object","of","a","reagent.core.parameters.EvaluationParameters","data","class","."]
code: def __init__(
      self,
      rl_parameters: RLParameters,
      metrics_to_score=None,
      actions: Optional[List[str]] = None,
      evaluation_parameters: Optional[EvaluationParameters] = None,
  ):
      """
      Args:
          rl_parameters: an object of a reagent.core.parameters.RLParameters
              data class.
          metrics_to_score: a list of strings representing extra reward
              metrics, excluding the reward itself.
          actions: a list of string action names for available actions.
          evaluation_parameters: an object of a
              reagent.core.parameters.EvaluationParameters data class.
      """
      super().__init__()
      self.rl_parameters = rl_parameters
      self.time_diff_unit_length = rl_parameters.time_diff_unit_length
      self.tensorboard_logging_freq = (
          rl_parameters.tensorboard_logging_freq
      )
      self.calc_cpe_in_training = (
          evaluation_parameters
          and evaluation_parameters.calc_cpe_in_training
      )
      assert actions is not None
      self._actions: List[str] = actions
  
      if rl_parameters.q_network_loss == "mse":
          self.q_network_loss = F.mse_loss
      elif rl_parameters.q_network_loss == "huber":
          self.q_network_loss = F.smooth_l1_loss
      else:
          raise Exception(
              "Q-Network loss type {} not valid loss.".format(
                  rl_parameters.q_network_loss
              )
          )
  
      if metrics_to_score:
          self.metrics_to_score = metrics_to_score + ["reward"]
      else:
          self.metrics_to_score = ["reward"]
  
      self._init_reward_boosts(rl_parameters.reward_boost)
  
imports: import logging
  from abc import abstractmethod
  from typing import Dict, List, Optional
  import reagent.core.types
  import torch
  import torch.nn.functional
  from reagent.core.parameters import EvaluationParameters, RLParameters
  from reagent.core.torch_utils import masked_softmax
  from reagent.evaluation.evaluation_data_page import EvaluationDataPage
  from reagent.evaluation.evaluator import Evaluator
  from reagent.optimizer import Optimizer__Union
  from reagent.training.reagent_lightning_module import ReAgentLightningModule
  from reagent.training.rl_trainer_pytorch import RLTrainerMixin
label_desc: Use image node_id 1 to create a new DQNTrainerBaseLightning object from inherited base classes: DQNTrainerMixin, RLTrainerMixin, ReAgentLightningModule with example: obj = DQNTrainerBaseLightning(rl_parameters, metrics_to_score, actions, evaluation_parameters)