class_docstr: An extension of Rainbow to perform implicit quantile regression.
docstr: Initializes the agent and constructs the Graph.
  
  Most of this constructor's parameters are IQN-specific hyperparameters whose
  values are taken from Dabney et al. (2018).
  
  Args:
    sess: `tf.compat.v1.Session` object for running associated ops.
    num_actions: int, number of actions the agent can take at any state.
    network: tf.Keras.Model, expects three parameters:
      (num_actions, quantile_embedding_dim, network_type). This class is used
      to generate network instances that are used by the agent. Each
      instantiation would have different set of variables. See
      dopamine.discrete_domains.atari_lib.NatureDQNNetwork as an example.
    kappa: float, Huber loss cutoff.
    num_tau_samples: int, number of online quantile samples for loss
      estimation.
    num_tau_prime_samples: int, number of target quantile samples for loss
      estimation.
    num_quantile_samples: int, number of quantile samples for computing
      Q-values.
    quantile_embedding_dim: int, embedding dimension for the quantile input.
    double_dqn: boolean, whether to perform double DQN style learning
      as described in Van Hasselt et al.: https://arxiv.org/abs/1509.06461.
    summary_writer: SummaryWriter object for outputting training statistics.
      Summary writing disabled if set to None.
    summary_writing_frequency: int, frequency with which summaries will be
      written. Lower values will result in slower training.
docstr_tok: ["Initializes","the","agent","and","constructs","the","Graph",".","Most","of","this","constructor","'s","parameters","are","IQN-specific","hyperparameters","whose","values","are","taken","from","Dabney","et","al",".","(","2018",")",".","Args",":","sess",":","`","tf.compat.v1.Session","`","object","for","running","associated","ops",".","num_actions",":","int",",","number","of","actions","the","agent","can","take","at","any","state",".","network",":","tf.Keras.Model",",","expects","three","parameters",":","(","num_actions",",","quantile_embedding_dim",",","network_type",")",".","This","class","is","used","to","generate","network","instances","that","are","used","by","the","agent",".","Each","instantiation","would","have","different","set","of","variables",".","See","dopamine.discrete_domains.atari_lib.NatureDQNNetwork","as","an","example",".","kappa",":","float",",","Huber","loss","cutoff",".","num_tau_samples",":","int",",","number","of","online","quantile","samples","for","loss","estimation",".","num_tau_prime_samples",":","int",",","number","of","target","quantile","samples","for","loss","estimation",".","num_quantile_samples",":","int",",","number","of","quantile","samples","for","computing","Q-values",".","quantile_embedding_dim",":","int",",","embedding","dimension","for","the","quantile","input",".","double_dqn",":","boolean",",","whether","to","perform","double","DQN","style","learning","as","described","in","Van","Hasselt","et","al",".",":","https",":","\/\/arxiv.org\/abs\/1509.06461",".","summary_writer",":","SummaryWriter","object","for","outputting","training","statistics",".","Summary","writing","disabled","if","set","to","None",".","summary_writing_frequency",":","int",",","frequency","with","which","summaries","will","be","written",".","Lower","values","will","result","in","slower","training","."]
code: def __init__(
      self,
      sess,
      num_actions,
      network=atari_lib.ImplicitQuantileNetwork,
      kappa=1.0,
      num_tau_samples=32,
      num_tau_prime_samples=32,
      num_quantile_samples=32,
      quantile_embedding_dim=64,
      double_dqn=False,
      summary_writer=None,
      summary_writing_frequency=500,
  ):
      """Initializes the agent and constructs the Graph.
  
      Most of this constructor's parameters are IQN-specific hyperparameters whose
      values are taken from Dabney et al. (2018).
  
      Args:
        sess: `tf.compat.v1.Session` object for running associated ops.
        num_actions: int, number of actions the agent can take at any state.
        network: tf.Keras.Model, expects three parameters:
          (num_actions, quantile_embedding_dim, network_type). This class is used
          to generate network instances that are used by the agent. Each
          instantiation would have different set of variables. See
          dopamine.discrete_domains.atari_lib.NatureDQNNetwork as an example.
        kappa: float, Huber loss cutoff.
        num_tau_samples: int, number of online quantile samples for loss
          estimation.
        num_tau_prime_samples: int, number of target quantile samples for loss
          estimation.
        num_quantile_samples: int, number of quantile samples for computing
          Q-values.
        quantile_embedding_dim: int, embedding dimension for the quantile input.
        double_dqn: boolean, whether to perform double DQN style learning
          as described in Van Hasselt et al.: https://arxiv.org/abs/1509.06461.
        summary_writer: SummaryWriter object for outputting training statistics.
          Summary writing disabled if set to None.
        summary_writing_frequency: int, frequency with which summaries will be
          written. Lower values will result in slower training.
      """
      self.kappa = kappa
      # num_tau_samples = N below equation (3) in the paper.
      self.num_tau_samples = num_tau_samples
      # num_tau_prime_samples = N' below equation (3) in the paper.
      self.num_tau_prime_samples = num_tau_prime_samples
      # num_quantile_samples = k below equation (3) in the paper.
      self.num_quantile_samples = num_quantile_samples
      # quantile_embedding_dim = n above equation (4) in the paper.
      self.quantile_embedding_dim = quantile_embedding_dim
      # option to perform double dqn.
      self.double_dqn = double_dqn
  
      super(ImplicitQuantileAgent, self).__init__(
          sess=sess,
          num_actions=num_actions,
          network=network,
          summary_writer=summary_writer,
          summary_writing_frequency=summary_writing_frequency,
      )
  
file_path: dopamine/dopamine/agents/implicit_quantile/implicit_quantile_agent.py
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  from dopamine.agents.rainbow import rainbow_agent
  from dopamine.discrete_domains import atari_lib
  import gin.tf
  import tensorflow
label_desc: Use image node_id 1 to create a new ImplicitQuantileAgent object from inherited base classes: rainbow_agent with example: obj = ImplicitQuantileAgent(sess, num_actions, network, kappa, num_tau_samples, num_tau_prime_samples, num_quantile_samples, quantile_embedding_dim, double_dqn, summary_writer, summary_writing_frequency)