class_docstr: Deep Q learning agent for episodic setting.
  
  A Q learning algorithm consists of several components:
  
      - A **Q-net** takes in a state and returns Q-value for action sampling.
        See :class:`~texar.tf.modules.CategoricalQNet` for an example Q-net
        class and required interface.
      - A **replay memory** manages past experience for Q-net updates. See        :class:`~texar.tf.core.DequeReplayMemory` for an example replay memory        class and required interface.
      - An **exploration** that specifies the exploration strategy used        to train the Q-net. See        :class:`~texar.tf.core.EpsilonLinearDecayExploration` for an example        class and required interface.
  
  Args:
      env_config: An instance of :class:`~texar.tf.agents.EnvConfig`
          specifying action space, observation space, and reward range, etc.
          Use :func:`~texar.tf.agents.get_gym_env_config` to create an
          EnvConfig from a gym environment.
      sess (optional): A tf session.
          Can be `None` here and set later with `agent.sess = session`.
      qnet (optional): A Q network that predicts Q values given states.
          If not given, a Q network is created based on :attr:`hparams`.
      target (optional): A target network to compute target Q values.
      qnet_kwargs (dict, optional): Keyword arguments for qnet
          constructor. Note that the `hparams` argument for network
          constructor is specified in the "policy_hparams" field of
          :attr:`hparams` and should not be included in `policy_kwargs`.
          Ignored if :attr:`qnet` is given.
      qnet_caller_kwargs (dict, optional): Keyword arguments for
          calling `qnet` to get Q values. The `qnet` is called with
          :python:`outputs=qnet(inputs=observation, **qnet_caller_kwargs)`
      replay_memory (optional): A replay memory instance.
          If not given, a replay memory is created based on :attr:`hparams`.
      replay_memory_kwargs (dict, optional): Keyword arguments for
          replay_memory constructor.
          Ignored if :attr:`replay_memory` is given.
      exploration (optional): An exploration instance used in the algorithm.
          If not given, an exploration instance is created based on
          :attr:`hparams`.
      exploration_kwargs (dict, optional): Keyword arguments for exploration
          class constructor. Ignored if :attr:`exploration` is given.
      hparams (dict or HParams, optional): Hyperparameters. Missing
          hyperparamerters will be set to default values. See
          :meth:`default_hparams` for the hyperparameter sturcture and
          default values.
class_docstr_tok: ["Deep","Q","learning","agent","for","episodic","setting",".","A","Q","learning","algorithm","consists","of","several","components",":","-","A","*","*","Q-net","*","*","takes","in","a","state","and","returns","Q-value","for","action","sampling",".","See",":","class",":","`","~texar.tf.modules.CategoricalQNet","`","for","an","example","Q-net","class","and","required","interface",".","-","A","*","*","replay","memory","*","*","manages","past","experience","for","Q-net","updates",".","See",":","class",":","`","~texar.tf.core.DequeReplayMemory","`","for","an","example","replay","memory","class","and","required","interface",".","-","An","*","*","exploration","*","*","that","specifies","the","exploration","strategy","used","to","train","the","Q-net",".","See",":","class",":","`","~texar.tf.core.EpsilonLinearDecayExploration","`","for","an","example","class","and","required","interface",".","Args",":","env_config",":","An","instance","of",":","class",":","`","~texar.tf.agents.EnvConfig","`","specifying","action","space",",","observation","space",",","and","reward","range",",","etc",".","Use",":","func",":","`","~texar.tf.agents.get_gym_env_config","`","to","create","an","EnvConfig","from","a","gym","environment",".","sess","(","optional",")",":","A","tf","session",".","Can","be","`","None","`","here","and","set","later","with","`","agent.sess","=","session","`",".","qnet","(","optional",")",":","A","Q","network","that","predicts","Q","values","given","states",".","If","not","given",",","a","Q","network","is","created","based","on",":","attr",":","`","hparams","`",".","target","(","optional",")",":","A","target","network","to","compute","target","Q","values",".","qnet_kwargs","(","dict",",","optional",")",":","Keyword","arguments","for","qnet","constructor",".","Note","that","the","`","hparams","`","argument","for","network","constructor","is","specified","in","the","``","policy_hparams","''","field","of",":","attr",":","`","hparams","`","and","should","not","be","included","in","`","policy_kwargs","`",".","Ignored","if",":","attr",":","`","qnet","`","is","given",".","qnet_caller_kwargs","(","dict",",","optional",")",":","Keyword","arguments","for","calling","`","qnet","`","to","get","Q","values",".","The","`","qnet","`","is","called","with",":","python",":","`","outputs=qnet","(","inputs=observation",",","*","*","qnet_caller_kwargs",")","`","replay_memory","(","optional",")",":","A","replay","memory","instance",".","If","not","given",",","a","replay","memory","is","created","based","on",":","attr",":","`","hparams","`",".","replay_memory_kwargs","(","dict",",","optional",")",":","Keyword","arguments","for","replay_memory","constructor",".","Ignored","if",":","attr",":","`","replay_memory","`","is","given",".","exploration","(","optional",")",":","An","exploration","instance","used","in","the","algorithm",".","If","not","given",",","an","exploration","instance","is","created","based","on",":","attr",":","`","hparams","`",".","exploration_kwargs","(","dict",",","optional",")",":","Keyword","arguments","for","exploration","class","constructor",".","Ignored","if",":","attr",":","`","exploration","`","is","given",".","hparams","(","dict","or","HParams",",","optional",")",":","Hyperparameters",".","Missing","hyperparamerters","will","be","set","to","default","values",".","See",":","meth",":","`","default_hparams","`","for","the","hyperparameter","sturcture","and","default","values","."]
docstr: Returns a dictionary of hyperparameters with default values:
  
  .. role:: python(code)
     :language: python
  
  .. code-block:: python
  
      {
          'qnet_type': 'CategoricalQNet',
          'qnet_hparams': None,
          'replay_memory_type': 'DequeReplayMemory',
          'replay_memory_hparams': None,
          'exploration_type': 'EpsilonLinearDecayExploration',
          'exploration_hparams': None,
          'optimization': opt.default_optimization_hparams(),
          'target_update_strategy': 'copy',
          'cold_start_steps': 100,
          'sample_batch_size': 32,
          'update_period': 100,
          'discount_factor': 0.95,
          'name': 'dqn_agent'
      }
  
  Here:
  
  "qnet_type": str or class or instance
      Q-value net. Can be class, its
      name or module path, or a class instance. If class name is given,
      the class must be from module :mod:`texar.tf.modules` or
      :mod:`texar.tf.custom`. Ignored if a `qnet` is given to
      the agent constructor.
  
  "qnet_hparams": dict, optional
      Hyperparameters for the Q net. With the :attr:`qnet_kwargs`
      argument to the constructor, a network is created with
      :python:`qnet_class(**qnet_kwargs, hparams=qnet_hparams)`.
  
  "replay_memory_type": str or class or instance
      Replay memory class. Can be class, its name or module path,
      or a class instance.
      If class name is given, the class must be from module
      :mod:`texar.tf.core` or :mod:`texar.tf.custom`.
      Ignored if a `replay_memory` is given to the agent constructor.
  
  "replay_memory_hparams": dict, optional
      Hyperparameters for the replay memory. With the
      :attr:`replay_memory_kwargs` argument to the constructor,
      a network is created with
      :python:`replay_memory_class(
      **replay_memory_kwargs, hparams=replay_memory_hparams)`.
  
  "exploration_type": str or class or instance
      Exploration class. Can be class,
      its name or module path, or a class instance. If class name is
      given, the class must be from module :mod:`texar.tf.core` or
      :mod:`texar.tf.custom`. Ignored if a `exploration` is given to
      the agent constructor.
  
  "exploration_hparams": dict, optional
      Hyperparameters for the exploration class.
      With the :attr:`exploration_kwargs` argument to the constructor,
      a network is created with :python:`exploration_class(
      **exploration_kwargs, hparams=exploration_hparams)`.
  
  "optimization": dict
      Hyperparameters of optimization for updating the Q-net.
      See :func:`~texar.tf.core.default_optimization_hparams` for details.
  
  "cold_start_steps": int
      In the beginning, Q-net is not trained in the first few steps.
  
  "sample_batch_size": int
      The number of samples taken in replay memory when training.
  
  "target_update_strategy": string
  
      - If **"copy"**, the target network is assigned with the parameter             of Q-net every :attr:`"update_period"` steps.
      - If **"tau"**, target will be updated by assigning as             ``` (1 - 1/update_period) * target + 1/update_period * qnet ```
  
  "update_period": int
      Frequecy of updating the target network, i.e., updating
      the target once for every "update_period" steps.
  
  "discount_factor": float
      The discount factor of reward.
  
  "name": str
      Name of the agent.
docstr_tok: ["Returns","a","dictionary","of","hyperparameters","with","default","values",":","..","role",":",":","python","(","code",")",":","language",":","python","..","code-block",":",":","python","{","'qnet_type","'",":","'CategoricalQNet","'",",","'qnet_hparams","'",":","None",",","'replay_memory_type","'",":","'DequeReplayMemory","'",",","'replay_memory_hparams","'",":","None",",","'exploration_type","'",":","'EpsilonLinearDecayExploration","'",",","'exploration_hparams","'",":","None",",","'optimization","'",":","opt.default_optimization_hparams","(",")",",","'target_update_strategy","'",":","'copy","'",",","'cold_start_steps","'",":","100",",","'sample_batch_size","'",":","32",",","'update_period","'",":","100",",","'discount_factor","'",":","0.95",",","'name","'",":","'dqn_agent'","}","Here",":","''","qnet_type","''",":","str","or","class","or","instance","Q-value","net",".","Can","be","class",",","its","name","or","module","path",",","or","a","class","instance",".","If","class","name","is","given",",","the","class","must","be","from","module",":","mod",":","`","texar.tf.modules","`","or",":","mod",":","`","texar.tf.custom","`",".","Ignored","if","a","`","qnet","`","is","given","to","the","agent","constructor",".","``","qnet_hparams","''",":","dict",",","optional","Hyperparameters","for","the","Q","net",".","With","the",":","attr",":","`","qnet_kwargs","`","argument","to","the","constructor",",","a","network","is","created","with",":","python",":","`","qnet_class","(","*","*","qnet_kwargs",",","hparams=qnet_hparams",")","`",".","``","replay_memory_type","''",":","str","or","class","or","instance","Replay","memory","class",".","Can","be","class",",","its","name","or","module","path",",","or","a","class","instance",".","If","class","name","is","given",",","the","class","must","be","from","module",":","mod",":","`","texar.tf.core","`","or",":","mod",":","`","texar.tf.custom","`",".","Ignored","if","a","`","replay_memory","`","is","given","to","the","agent","constructor",".","``","replay_memory_hparams","''",":","dict",",","optional","Hyperparameters","for","the","replay","memory",".","With","the",":","attr",":","`","replay_memory_kwargs","`","argument","to","the","constructor",",","a","network","is","created","with",":","python",":","`","replay_memory_class","(","*","*","replay_memory_kwargs",",","hparams=replay_memory_hparams",")","`",".","``","exploration_type","''",":","str","or","class","or","instance","Exploration","class",".","Can","be","class",",","its","name","or","module","path",",","or","a","class","instance",".","If","class","name","is","given",",","the","class","must","be","from","module",":","mod",":","`","texar.tf.core","`","or",":","mod",":","`","texar.tf.custom","`",".","Ignored","if","a","`","exploration","`","is","given","to","the","agent","constructor",".","``","exploration_hparams","''",":","dict",",","optional","Hyperparameters","for","the","exploration","class",".","With","the",":","attr",":","`","exploration_kwargs","`","argument","to","the","constructor",",","a","network","is","created","with",":","python",":","`","exploration_class","(","*","*","exploration_kwargs",",","hparams=exploration_hparams",")","`",".","``","optimization","''",":","dict","Hyperparameters","of","optimization","for","updating","the","Q-net",".","See",":","func",":","`","~texar.tf.core.default_optimization_hparams","`","for","details",".","``","cold_start_steps","''",":","int","In","the","beginning",",","Q-net","is","not","trained","in","the","first","few","steps",".","``","sample_batch_size","''",":","int","The","number","of","samples","taken","in","replay","memory","when","training",".","``","target_update_strategy","''",":","string","-","If","*","*","''","copy","''","*","*",",","the","target","network","is","assigned","with","the","parameter","of","Q-net","every",":","attr",":","`","``","update_period","''","`","steps",".","-","If","*","*","''","tau","''","*","*",",","target","will","be","updated","by","assigning","as","``","`","(","1","-","1\/update_period",")","*","target","+","1\/update_period","*","qnet","``","`","''","update_period","''",":","int","Frequecy","of","updating","the","target","network",",","i.e.",",","updating","the","target","once","for","every","``","update_period","''","steps",".","``","discount_factor","''",":","float","The","discount","factor","of","reward",".","``","name","''",":","str","Name","of","the","agent","."]
code: def default_hparams():
      """Returns a dictionary of hyperparameters with default values:
  
          .. role:: python(code)
             :language: python
  
          .. code-block:: python
  
              {
                  'qnet_type': 'CategoricalQNet',
                  'qnet_hparams': None,
                  'replay_memory_type': 'DequeReplayMemory',
                  'replay_memory_hparams': None,
                  'exploration_type': 'EpsilonLinearDecayExploration',
                  'exploration_hparams': None,
                  'optimization': opt.default_optimization_hparams(),
                  'target_update_strategy': 'copy',
                  'cold_start_steps': 100,
                  'sample_batch_size': 32,
                  'update_period': 100,
                  'discount_factor': 0.95,
                  'name': 'dqn_agent'
              }
  
          Here:
  
          "qnet_type": str or class or instance
              Q-value net. Can be class, its
              name or module path, or a class instance. If class name is given,
              the class must be from module :mod:`texar.tf.modules` or
              :mod:`texar.tf.custom`. Ignored if a `qnet` is given to
              the agent constructor.
  
          "qnet_hparams": dict, optional
              Hyperparameters for the Q net. With the :attr:`qnet_kwargs`
              argument to the constructor, a network is created with
              :python:`qnet_class(**qnet_kwargs, hparams=qnet_hparams)`.
  
          "replay_memory_type": str or class or instance
              Replay memory class. Can be class, its name or module path,
              or a class instance.
              If class name is given, the class must be from module
              :mod:`texar.tf.core` or :mod:`texar.tf.custom`.
              Ignored if a `replay_memory` is given to the agent constructor.
  
          "replay_memory_hparams": dict, optional
              Hyperparameters for the replay memory. With the
              :attr:`replay_memory_kwargs` argument to the constructor,
              a network is created with
              :python:`replay_memory_class(
              **replay_memory_kwargs, hparams=replay_memory_hparams)`.
  
          "exploration_type": str or class or instance
              Exploration class. Can be class,
              its name or module path, or a class instance. If class name is
              given, the class must be from module :mod:`texar.tf.core` or
              :mod:`texar.tf.custom`. Ignored if a `exploration` is given to
              the agent constructor.
  
          "exploration_hparams": dict, optional
              Hyperparameters for the exploration class.
              With the :attr:`exploration_kwargs` argument to the constructor,
              a network is created with :python:`exploration_class(
              **exploration_kwargs, hparams=exploration_hparams)`.
  
          "optimization": dict
              Hyperparameters of optimization for updating the Q-net.
              See :func:`~texar.tf.core.default_optimization_hparams` for details.
  
          "cold_start_steps": int
              In the beginning, Q-net is not trained in the first few steps.
  
          "sample_batch_size": int
              The number of samples taken in replay memory when training.
  
          "target_update_strategy": string
  
              - If **"copy"**, the target network is assigned with the parameter \
              of Q-net every :attr:`"update_period"` steps.
              - If **"tau"**, target will be updated by assigning as \
              ``` (1 - 1/update_period) * target + 1/update_period * qnet ```
  
          "update_period": int
              Frequecy of updating the target network, i.e., updating
              the target once for every "update_period" steps.
  
          "discount_factor": float
              The discount factor of reward.
  
          "name": str
              Name of the agent.
          """
      return {
          "qnet_type": "CategoricalQNet",
          "qnet_hparams": None,
          "replay_memory_type": "DequeReplayMemory",
          "replay_memory_hparams": None,
          "exploration_type": "EpsilonLinearDecayExploration",
          "exploration_hparams": None,
          "optimization": opt.default_optimization_hparams(),
          "target_update_strategy": "copy",
          "cold_start_steps": 100,
          "sample_batch_size": 32,
          "update_period": 100,
          "discount_factor": 0.95,
          "name": "dqn_agent",
      }
  
imports: import random
  import numpy
  import tensorflow
  from texar.tf import context
  from texar.tf.agents.episodic_agent_base import EpisodicAgentBase
  from texar.tf.utils import utils
  from texar.tf.core import optimization
label_desc: Use image node_id 2 for calling the DQNAgent obj's underlying member method code with example usage: obj.default_hparams() and returns: dict