class_docstr: Tensorforce agent (specification key: `tensorforce`).
  
  Highly configurable agent and basis for a broad class of deep reinforcement learning agents,
  which act according to a policy parametrized by a neural network, leverage a memory module for
  periodic updates based on batches of experience, and optionally employ a baseline/critic/target
  policy for improved reward estimation.
  
  Args:
      states (specification): States specification
          (<span style="color:#C00000"><b>required</b></span>, better implicitly specified via
          `environment` argument for `Agent.create()`), arbitrarily nested dictionary of state
          descriptions (usually taken from `Environment.states()`) with the following attributes:
          <ul>
          <li><b>type</b> (<i>"bool" | "int" | "float"</i>) &ndash; state data type
          (<span style="color:#00C000"><b>default</b></span>: "float").</li>
          <li><b>shape</b> (<i>int | iter[int]</i>) &ndash; state shape
          (<span style="color:#C00000"><b>required</b></span>).</li>
          <li><b>num_values</b> (<i>int > 0</i>) &ndash; number of discrete state values
          (<span style="color:#C00000"><b>required</b></span> for type "int").</li>
          <li><b>min_value/max_value</b> (<i>float</i>) &ndash; minimum/maximum state value
          (<span style="color:#00C000"><b>optional</b></span> for type "float").</li>
          </ul>
      actions (specification): Actions specification
          (<span style="color:#C00000"><b>required</b></span>, better implicitly specified via
          `environment` argument for `Agent.create()`), arbitrarily nested dictionary of
          action descriptions (usually taken from `Environment.actions()`) with the following
          attributes:
          <ul>
          <li><b>type</b> (<i>"bool" | "int" | "float"</i>) &ndash; action data type
          (<span style="color:#C00000"><b>required</b></span>).</li>
          <li><b>shape</b> (<i>int > 0 | iter[int > 0]</i>) &ndash; action shape
          (<span style="color:#00C000"><b>default</b></span>: scalar).</li>
          <li><b>num_values</b> (<i>int > 0</i>) &ndash; number of discrete action values
          (<span style="color:#C00000"><b>required</b></span> for type "int").</li>
          <li><b>min_value/max_value</b> (<i>float</i>) &ndash; minimum/maximum action value
          (<span style="color:#00C000"><b>optional</b></span> for type "float").</li>
          </ul>
      max_episode_timesteps (int > 0): Upper bound for numer of timesteps per episode
          (<span style="color:#00C000"><b>default</b></span>: not given, better implicitly
          specified via `environment` argument for `Agent.create()`).
  
      policy (specification): Policy configuration, see [networks](../modules/networks.html) and
          [policies documentation](../modules/policies.html)
          (<span style="color:#00C000"><b>default</b></span>: action distributions or value
          functions parametrized by an automatically configured network).
      memory (int | specification): Replay memory capacity, or memory configuration, see the
          [memories documentation](../modules/memories.html)
          (<span style="color:#00C000"><b>default</b></span>: minimum capacity recent memory).
      update (int | specification): Model update configuration with the following attributes
          (<span style="color:#C00000"><b>required</b>,
          <span style="color:#00C000"><b>default</b></span>: timesteps batch size</span>):
          <ul>
          <li><b>unit</b> (<i>"timesteps" | "episodes"</i>) &ndash; unit for update attributes
          (<span style="color:#C00000"><b>required</b></span>).</li>
          <li><b>batch_size</b>
          (<i><a href="../modules/parameters.html">parameter</a>, int > 0</i>) &ndash;
          size of update batch in number of units
          (<span style="color:#C00000"><b>required</b></span>).</li>
          <li><b>frequency</b>
          (<i>"never" | <a href="../modules/parameters.html">parameter</a>, int > 0 | 0.0 < float <= 1.0</i>) &ndash;
          frequency of updates, relative to batch_size if float
          (<span style="color:#00C000"><b>default</b></span>: batch_size).</li>
          <li><b>start</b>
          (<i><a href="../modules/parameters.html">parameter</a>, int >= batch_size</i>) &ndash;
          number of units before first update
          (<span style="color:#00C000"><b>default</b></span>: none).</li>
          </ul>
      optimizer (specification): Optimizer configuration, see the
          [optimizers documentation](../modules/optimizers.html)
          (<span style="color:#00C000"><b>default</b></span>: Adam optimizer).
      objective (specification): Optimization objective configuration, see the
          [objectives documentation](../modules/objectives.html)
          (<span style="color:#C00000"><b>required</b></span>).
      reward_estimation (specification): Reward estimation configuration with the following
          attributes (<span style="color:#C00000"><b>required</b></span>):
          <ul>
          <li><b>horizon</b>
          (<i>"episode" | <a href="../modules/parameters.html">parameter</a>, int >= 1</i>)
          &ndash; Horizon of discounted-sum return estimation
          (<span style="color:#C00000"><b>required</b></span>).</li>
          <li><b>discount</b>
          (<i><a href="../modules/parameters.html">parameter</a>, 0.0 <= float <= 1.0</i>) &ndash;
          Discount factor of future rewards for discounted-sum return estimation
          (<span style="color:#00C000"><b>default</b></span>: 1.0).</li>
          <li><b>predict_horizon_values</b> (<i>false | "early" | "late"</i>) &ndash; Whether to
          include a baseline prediction of the horizon value as part of the return estimation, and
          if so, whether to compute the horizon value prediction "early" when experiences are
          stored to memory, or "late" when batches of experience are retrieved for the update
          (<span style="color:#00C000"><b>default</b></span>: "late" if baseline_policy or
          baseline_objective are specified, else false).</li>
          <li><b>estimate_advantage</b> (<i>False | "early" | "late"</i>) &ndash; Whether to use
          an estimate of the advantage (return minus baseline value prediction) instead of the
          return as learning signal, and whether to do so late after the baseline update
          (default) or early before the baseline update
          (<span style="color:#00C000"><b>default</b></span>: false, unless baseline_policy is
          specified but baseline_objective/optimizer are not).</li>
          <li><b>predict_action_values</b> (<i>bool</i>) &ndash; Whether to predict state-action-
          instead of state-values as horizon values and for advantage estimation
          (<span style="color:#00C000"><b>default</b></span>: false).</li>
          <li><b>reward_processing</b> (<i>specification)</i>) &ndash; Reward preprocessing as
          layer or list of layers, see the
          [preprocessing documentation](../modules/preprocessing.html)
          (<span style="color:#00C000"><b>default</b></span>: no reward processing).</li>
          <li><b>return_processing</b> (<i>specification</i>) &ndash; Return processing as layer
          or list of layers, see the [preprocessing documentation](../modules/preprocessing.html)
          (<span style="color:#00C000"><b>default</b></span>: no return processing).</li>
          <li><b>advantage_processing</b> (<i>specification</i>) &ndash; Advantage processing as
          layer or list of layers, see the [preprocessing documentation](../modules/preprocessing.html)
          (<span style="color:#00C000"><b>default</b></span>: no advantage processing).</li>
          <li><b>predict_terminal_values</b> (<i>bool</i>) &ndash; Whether to predict the value
          of terminal states, usually not required since max_episode_timesteps terminals are
          handled separately
          (<span style="color:#00C000"><b>default</b></span>: false).</li>
          </ul>
  
      baseline (specification): Baseline configuration, policy will be used as baseline if none,
          see [networks](../modules/networks.html) and potentially
          [policies documentation](../modules/policies.html)
          (<span style="color:#00C000"><b>default</b></span>: none).
      baseline_optimizer (specification | <a href="../modules/parameters.html">parameter</a>, float > 0.0):
          Baseline optimizer configuration, see the
          [optimizers documentation](../modules/optimizers.html),
          main optimizer will be used for baseline if none, a float implies none and specifies a
          custom weight for the baseline loss
          (<span style="color:#00C000"><b>default</b></span>: none).
      baseline_objective (specification): Baseline optimization objective configuration, see the
          [objectives documentation](../modules/objectives.html),
          required if baseline optimizer is specified, main objective will be used for baseline if
          baseline objective and optimizer are not specified
          (<span style="color:#00C000"><b>default</b></span>: none).
  
      l2_regularization (<a href="../modules/parameters.html">parameter</a>, float >= 0.0):
          L2 regularization loss weight
          (<span style="color:#00C000"><b>default</b></span>: no L2 regularization).
      entropy_regularization (<a href="../modules/parameters.html">parameter</a>, float >= 0.0):
          Entropy regularization loss weight, to discourage the policy distribution from being
          "too certain"
          (<span style="color:#00C000"><b>default</b></span>: no entropy regularization).
  
      state_preprocessing (dict[specification]): State preprocessing as layer or list of layers,
          see the [preprocessing documentation](../modules/preprocessing.html),
          specified per state-type or -name
          (<span style="color:#00C000"><b>default</b></span>: linear normalization of bounded
          float states to [-2.0, 2.0]).
      exploration (<a href="../modules/parameters.html">parameter</a> | dict[<a href="../modules/parameters.html">parameter</a>], float >= 0.0):
          Exploration, defined as the probability for uniformly random output in case of `bool`
          and `int` actions, and the standard deviation of Gaussian noise added to every output in
          case of `float` actions, specified globally or per action-type or -name
          (<span style="color:#00C000"><b>default</b></span>: no exploration).
      variable_noise (<a href="../modules/parameters.html">parameter</a>, float >= 0.0):
          Add Gaussian noise with given standard deviation to all trainable variables, as
          alternative exploration mechanism
          (<span style="color:#00C000"><b>default</b></span>: no variable noise).
  
      parallel_interactions (int > 0): Maximum number of parallel interactions to support,
          for instance, to enable multiple parallel episodes, environments or agents within an
          environment
          (<span style="color:#00C000"><b>default</b></span>: 1).
      config (specification): Additional configuration options:
          <ul>
          <li><b>name</b> (<i>string</i>) &ndash; Agent name, used e.g. for TensorFlow scopes and
          saver default filename
          (<span style="color:#00C000"><b>default</b></span>: "agent").
          <li><b>device</b> (<i>string</i>) &ndash; Device name
          (<span style="color:#00C000"><b>default</b></span>: CPU). Different from (un)supervised
          deep learning, RL does not always benefit from running on a GPU, depending on
          environment and agent configuration. In particular for RL-typical environments with
          low-dimensional state spaces (i.e., no images), one usually gets better performance by
          running on CPU only. Consequently, Tensorforce is configured to run on CPU by default,
          which can be changed, for instance, by setting this value to 'GPU' instead.
          <li><b>seed</b> (<i>int</i>) &ndash; Random seed to set for Python, NumPy (both set
          globally!) and TensorFlow, environment seed may have to be set separately for fully
          deterministic execution, generally not recommended since results in a fully
          deterministic setting are less meaningful/representative
          (<span style="color:#00C000"><b>default</b></span>: none).</li>
          <li><b>buffer_observe</b> (<i>false | "episode" | int > 0</i>) &ndash; Number of
          timesteps within an episode to buffer before calling the internal observe function, to
          reduce calls to TensorFlow for improved performance
          (<span style="color:#00C000"><b>default</b></span>: configuration-specific maximum
          number which can be buffered without affecting performance).</li>
          <li><b>enable_int_action_masking</b> (<i>bool</i>) &ndash; Whether int action options
          can be masked via an optional "[ACTION-NAME]_mask" state input
          (<span style="color:#00C000"><b>default</b></span>: true).</li>
          <li><b>create_tf_assertions</b> (<i>bool</i>) &ndash; Whether to create internal
          TensorFlow assertion operations
          (<span style="color:#00C000"><b>default</b></span>: true).</li>
          <li><b>eager_mode</b> (<i>bool</i>) &ndash; Whether to run functions eagerly instead of
          running as a traced graph function, can be helpful for debugging
          (<span style="color:#00C000"><b>default</b></span>: false).</li>
          <li><b>tf_log_level</b> (<i>int >= 0</i>) &ndash; TensorFlow log level, additional C++
          logging messages can be enabled by setting os.environ["TF_CPP_MIN_LOG_LEVEL"] = "1"/"2"
          before importing Tensorforce/TensorFlow
          (<span style="color:#00C000"><b>default</b></span>: 40, only error and critical).</li>
          </ul>
      saver (path | specification): TensorFlow checkpoints directory, or checkpoint manager
          configuration with the following attributes, for periodic implicit saving as alternative
          to explicit saving via agent.save()
          (<span style="color:#00C000"><b>default</b></span>: no saver):
          <ul>
          <li><b>directory</b> (<i>path</i>) &ndash; checkpoint directory
          (<span style="color:#C00000"><b>required</b></span>).</li>
          <li><b>filename</b> (<i>string</i>) &ndash; checkpoint filename
          (<span style="color:#00C000"><b>default</b></span>: agent name).</li>
          <li><b>frequency</b> (<i>int > 0</i>) &ndash; how frequently to save a checkpoint
          (<span style="color:#C00000"><b>required</b></span>).</li>
          <li><b>unit</b> (<i>"timesteps" | "episodes" | "updates"</i>) &ndash; frequency unit
          (<span style="color:#00C000"><b>default</b></span>: updates).</li>
          <li><b>max_checkpoints</b> (<i>int > 0</i>) &ndash; maximum number of checkpoints to
          keep (<span style="color:#00C000"><b>default</b></span>: 10).</li>
          <li><b>max_hour_frequency</b> (<i>int > 0</i>) &ndash; ignoring max-checkpoints,
          definitely keep a checkpoint in given hour frequency
          (<span style="color:#00C000"><b>default</b></span>: none).</li>
          </ul>
      summarizer (path | specification): TensorBoard summaries directory, or summarizer
          configuration with the following attributes
          (<span style="color:#00C000"><b>default</b></span>: no summarizer):
          <ul>
          <li><b>directory</b> (<i>path</i>) &ndash; summarizer directory
          (<span style="color:#C00000"><b>required</b></span>).</li>
          <li><b>filename</b> (<i>path</i>) &ndash; summarizer filename, max_summaries does not
          apply if name specified
          (<span style="color:#00C000"><b>default</b></span>: "summary-%Y%m%d-%H%M%S").</li>
          <li><b>max_summaries</b> (<i>int > 0</i>) &ndash; maximum number of (generically-named)
          summaries to keep
          (<span style="color:#00C000"><b>default</b></span>: 7, number of different colors in
          Tensorboard).</li>
          <li><b>flush</b> (<i>int > 0</i>) &ndash; how frequently in seconds to flush the
          summary writer (<span style="color:#00C000"><b>default</b></span>: 10).</li>
          <li><b>summaries</b> (<i>"all" | iter[string]</i>) &ndash; which summaries to record,
          "all" implies all numerical summaries, so all summaries except "graph"
          (<span style="color:#00C000"><b>default</b></span>: "all"):</li>
          <li>"action-value": value of each action (timestep-based)</li>
          <li>"distribution": distribution parameters like probabilities or mean and stddev
          (timestep-based)</li>
          <li>"entropy": entropy of (per-action) policy distribution(s) (timestep-based)</li>
          <li>"graph": computation graph</li>
          <li>"kl-divergence": KL-divergence of previous and updated (per-action) policy
          distribution(s) (update-based)</li>
          <li>"loss": policy and baseline loss plus loss components (update-based)</li>
          <li>"parameters": parameter values (according to parameter unit)</li>
          <li>"reward": reward per timestep, episode length and reward, plus intermediate
          reward/return/advantage estimates and processed values
          (timestep/episode/update-based)</li>
          <li>"update-norm": global norm of update (update-based)</li>
          <li>"updates": mean and variance of update tensors per variable (update-based)</li>
          <li>"variables": mean of trainable variables tensors (update-based)</li>
          </ul>
      tracking ("all" | iter[string]): Which tensors to track, available values are a subset of
          the values of summarizer[summaries] above
          (<span style="color:#00C000"><b>default</b></span>: no tracking).
          The current value of tracked tensors can be retrieved via tracked_tensors() at any time,
          however, note that tensor values change at different timescales (timesteps, episodes,
          updates).
      recorder (path | specification): Traces recordings directory, or recorder configuration with
          the following attributes (see
          [record-and-pretrain script](https://github.com/tensorforce/tensorforce/blob/master/examples/record_and_pretrain.py)
          for example application)
          (<span style="color:#00C000"><b>default</b></span>: no recorder):
          <ul>
          <li><b>directory</b> (<i>path</i>) &ndash; recorder directory
          (<span style="color:#C00000"><b>required</b></span>).</li>
          <li><b>frequency</b> (<i>int > 0</i>) &ndash; how frequently in episodes to record
          traces (<span style="color:#00C000"><b>default</b></span>: every episode).</li>
          <li><b>start</b> (<i>int >= 0</i>) &ndash; how many episodes to skip before starting to
          record traces (<span style="color:#00C000"><b>default</b></span>: 0).</li>
          <li><b>max-traces</b> (<i>int > 0</i>) &ndash; maximum number of traces to keep
          (<span style="color:#00C000"><b>default</b></span>: all).</li>
class_docstr_tok: ["Tensorforce","agent","(","specification","key",":","`","tensorforce","`",")",".","Highly","configurable","agent","and","basis","for","a","broad","class","of","deep","reinforcement","learning","agents",",","which","act","according","to","a","policy","parametrized","by","a","neural","network",",","leverage","a","memory","module","for","periodic","updates","based","on","batches","of","experience",",","and","optionally","employ","a","baseline\/critic\/target","policy","for","improved","reward","estimation",".","Args",":","states","(","specification",")",":","States","specification","(","<","span","style=","''","color",":","#","C00000","''",">","<","b",">","required","<","\/b",">","<","\/span",">",",","better","implicitly","specified","via","`","environment","`","argument","for","`","Agent.create","(",")","`",")",",","arbitrarily","nested","dictionary","of","state","descriptions","(","usually","taken","from","`","Environment.states","(",")","`",")","with","the","following","attributes",":","<","ul",">","<","li",">","<","b",">","type","<","\/b",">","(","<","i",">","''","bool","''","|","``","int","''","|","``","float","''","<","\/i",">",")","&","ndash",";","state","data","type","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","``","float","''",")",".","<","\/li",">","<","li",">","<","b",">","shape","<","\/b",">","(","<","i",">","int","|","iter","[","int","]","<","\/i",">",")","&","ndash",";","state","shape","(","<","span","style=","''","color",":","#","C00000","''",">","<","b",">","required","<","\/b",">","<","\/span",">",")",".","<","\/li",">","<","li",">","<","b",">","num_values","<","\/b",">","(","<","i",">","int",">","0","<","\/i",">",")","&","ndash",";","number","of","discrete","state","values","(","<","span","style=","''","color",":","#","C00000","''",">","<","b",">","required","<","\/b",">","<","\/span",">","for","type","``","int","''",")",".","<","\/li",">","<","li",">","<","b",">","min_value\/max_value","<","\/b",">","(","<","i",">","float","<","\/i",">",")","&","ndash",";","minimum\/maximum","state","value","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","optional","<","\/b",">","<","\/span",">","for","type","``","float","''",")",".","<","\/li",">","<","\/ul",">","actions","(","specification",")",":","Actions","specification","(","<","span","style=","''","color",":","#","C00000","''",">","<","b",">","required","<","\/b",">","<","\/span",">",",","better","implicitly","specified","via","`","environment","`","argument","for","`","Agent.create","(",")","`",")",",","arbitrarily","nested","dictionary","of","action","descriptions","(","usually","taken","from","`","Environment.actions","(",")","`",")","with","the","following","attributes",":","<","ul",">","<","li",">","<","b",">","type","<","\/b",">","(","<","i",">","''","bool","''","|","``","int","''","|","``","float","''","<","\/i",">",")","&","ndash",";","action","data","type","(","<","span","style=","''","color",":","#","C00000","''",">","<","b",">","required","<","\/b",">","<","\/span",">",")",".","<","\/li",">","<","li",">","<","b",">","shape","<","\/b",">","(","<","i",">","int",">","0","|","iter","[","int",">","0","]","<","\/i",">",")","&","ndash",";","action","shape","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","scalar",")",".","<","\/li",">","<","li",">","<","b",">","num_values","<","\/b",">","(","<","i",">","int",">","0","<","\/i",">",")","&","ndash",";","number","of","discrete","action","values","(","<","span","style=","''","color",":","#","C00000","''",">","<","b",">","required","<","\/b",">","<","\/span",">","for","type","``","int","''",")",".","<","\/li",">","<","li",">","<","b",">","min_value\/max_value","<","\/b",">","(","<","i",">","float","<","\/i",">",")","&","ndash",";","minimum\/maximum","action","value","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","optional","<","\/b",">","<","\/span",">","for","type","``","float","''",")",".","<","\/li",">","<","\/ul",">","max_episode_timesteps","(","int",">","0",")",":","Upper","bound","for","numer","of","timesteps","per","episode","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","not","given",",","better","implicitly","specified","via","`","environment","`","argument","for","`","Agent.create","(",")","`",")",".","policy","(","specification",")",":","Policy","configuration",",","see","[","networks","]","(","..","\/modules\/networks.html",")","and","[","policies","documentation","]","(","..","\/modules\/policies.html",")","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","action","distributions","or","value","functions","parametrized","by","an","automatically","configured","network",")",".","memory","(","int","|","specification",")",":","Replay","memory","capacity",",","or","memory","configuration",",","see","the","[","memories","documentation","]","(","..","\/modules\/memories.html",")","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","minimum","capacity","recent","memory",")",".","update","(","int","|","specification",")",":","Model","update","configuration","with","the","following","attributes","(","<","span","style=","''","color",":","#","C00000","''",">","<","b",">","required","<","\/b",">",",","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","timesteps","batch","size","<","\/span",">",")",":","<","ul",">","<","li",">","<","b",">","unit","<","\/b",">","(","<","i",">","''","timesteps","''","|","``","episodes","''","<","\/i",">",")","&","ndash",";","unit","for","update","attributes","(","<","span","style=","''","color",":","#","C00000","''",">","<","b",">","required","<","\/b",">","<","\/span",">",")",".","<","\/li",">","<","li",">","<","b",">","batch_size","<","\/b",">","(","<","i",">","<","a","href=","''","..","\/modules\/parameters.html","''",">","parameter","<","\/a",">",",","int",">","0","<","\/i",">",")","&","ndash",";","size","of","update","batch","in","number","of","units","(","<","span","style=","''","color",":","#","C00000","''",">","<","b",">","required","<","\/b",">","<","\/span",">",")",".","<","\/li",">","<","li",">","<","b",">","frequency","<","\/b",">","(","<","i",">","''","never","''","|","<","a","href=","''","..","\/modules\/parameters.html","''",">","parameter","<","\/a",">",",","int",">","0","|","0.0","<","float","<","=","1.0","<","\/i",">",")","&","ndash",";","frequency","of","updates",",","relative","to","batch_size","if","float","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","batch_size",")",".","<","\/li",">","<","li",">","<","b",">","start","<","\/b",">","(","<","i",">","<","a","href=","''","..","\/modules\/parameters.html","''",">","parameter","<","\/a",">",",","int",">","=","batch_size","<","\/i",">",")","&","ndash",";","number","of","units","before","first","update","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","none",")",".","<","\/li",">","<","\/ul",">","optimizer","(","specification",")",":","Optimizer","configuration",",","see","the","[","optimizers","documentation","]","(","..","\/modules\/optimizers.html",")","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","Adam","optimizer",")",".","objective","(","specification",")",":","Optimization","objective","configuration",",","see","the","[","objectives","documentation","]","(","..","\/modules\/objectives.html",")","(","<","span","style=","''","color",":","#","C00000","''",">","<","b",">","required","<","\/b",">","<","\/span",">",")",".","reward_estimation","(","specification",")",":","Reward","estimation","configuration","with","the","following","attributes","(","<","span","style=","''","color",":","#","C00000","''",">","<","b",">","required","<","\/b",">","<","\/span",">",")",":","<","ul",">","<","li",">","<","b",">","horizon","<","\/b",">","(","<","i",">","''","episode","''","|","<","a","href=","''","..","\/modules\/parameters.html","''",">","parameter","<","\/a",">",",","int",">","=","1","<","\/i",">",")","&","ndash",";","Horizon","of","discounted-sum","return","estimation","(","<","span","style=","''","color",":","#","C00000","''",">","<","b",">","required","<","\/b",">","<","\/span",">",")",".","<","\/li",">","<","li",">","<","b",">","discount","<","\/b",">","(","<","i",">","<","a","href=","''","..","\/modules\/parameters.html","''",">","parameter","<","\/a",">",",","0.0","<","=","float","<","=","1.0","<","\/i",">",")","&","ndash",";","Discount","factor","of","future","rewards","for","discounted-sum","return","estimation","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","1.0",")",".","<","\/li",">","<","li",">","<","b",">","predict_horizon_values","<","\/b",">","(","<","i",">","false","|","``","early","''","|","``","late","''","<","\/i",">",")","&","ndash",";","Whether","to","include","a","baseline","prediction","of","the","horizon","value","as","part","of","the","return","estimation",",","and","if","so",",","whether","to","compute","the","horizon","value","prediction","``","early","''","when","experiences","are","stored","to","memory",",","or","``","late","''","when","batches","of","experience","are","retrieved","for","the","update","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","``","late","''","if","baseline_policy","or","baseline_objective","are","specified",",","else","false",")",".","<","\/li",">","<","li",">","<","b",">","estimate_advantage","<","\/b",">","(","<","i",">","False","|","``","early","''","|","``","late","''","<","\/i",">",")","&","ndash",";","Whether","to","use","an","estimate","of","the","advantage","(","return","minus","baseline","value","prediction",")","instead","of","the","return","as","learning","signal",",","and","whether","to","do","so","late","after","the","baseline","update","(","default",")","or","early","before","the","baseline","update","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","false",",","unless","baseline_policy","is","specified","but","baseline_objective\/optimizer","are","not",")",".","<","\/li",">","<","li",">","<","b",">","predict_action_values","<","\/b",">","(","<","i",">","bool","<","\/i",">",")","&","ndash",";","Whether","to","predict","state-action-","instead","of","state-values","as","horizon","values","and","for","advantage","estimation","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","false",")",".","<","\/li",">","<","li",">","<","b",">","reward_processing","<","\/b",">","(","<","i",">","specification",")","<","\/i",">",")","&","ndash",";","Reward","preprocessing","as","layer","or","list","of","layers",",","see","the","[","preprocessing","documentation","]","(","..","\/modules\/preprocessing.html",")","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","no","reward","processing",")",".","<","\/li",">","<","li",">","<","b",">","return_processing","<","\/b",">","(","<","i",">","specification","<","\/i",">",")","&","ndash",";","Return","processing","as","layer","or","list","of","layers",",","see","the","[","preprocessing","documentation","]","(","..","\/modules\/preprocessing.html",")","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","no","return","processing",")",".","<","\/li",">","<","li",">","<","b",">","advantage_processing","<","\/b",">","(","<","i",">","specification","<","\/i",">",")","&","ndash",";","Advantage","processing","as","layer","or","list","of","layers",",","see","the","[","preprocessing","documentation","]","(","..","\/modules\/preprocessing.html",")","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","no","advantage","processing",")",".","<","\/li",">","<","li",">","<","b",">","predict_terminal_values","<","\/b",">","(","<","i",">","bool","<","\/i",">",")","&","ndash",";","Whether","to","predict","the","value","of","terminal","states",",","usually","not","required","since","max_episode_timesteps","terminals","are","handled","separately","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","false",")",".","<","\/li",">","<","\/ul",">","baseline","(","specification",")",":","Baseline","configuration",",","policy","will","be","used","as","baseline","if","none",",","see","[","networks","]","(","..","\/modules\/networks.html",")","and","potentially","[","policies","documentation","]","(","..","\/modules\/policies.html",")","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","none",")",".","baseline_optimizer","(","specification","|","<","a","href=","''","..","\/modules\/parameters.html","''",">","parameter","<","\/a",">",",","float",">","0.0",")",":","Baseline","optimizer","configuration",",","see","the","[","optimizers","documentation","]","(","..","\/modules\/optimizers.html",")",",","main","optimizer","will","be","used","for","baseline","if","none",",","a","float","implies","none","and","specifies","a","custom","weight","for","the","baseline","loss","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","none",")",".","baseline_objective","(","specification",")",":","Baseline","optimization","objective","configuration",",","see","the","[","objectives","documentation","]","(","..","\/modules\/objectives.html",")",",","required","if","baseline","optimizer","is","specified",",","main","objective","will","be","used","for","baseline","if","baseline","objective","and","optimizer","are","not","specified","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","none",")",".","l2_regularization","(","<","a","href=","''","..","\/modules\/parameters.html","''",">","parameter","<","\/a",">",",","float",">","=","0.0",")",":","L2","regularization","loss","weight","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","no","L2","regularization",")",".","entropy_regularization","(","<","a","href=","''","..","\/modules\/parameters.html","''",">","parameter","<","\/a",">",",","float",">","=","0.0",")",":","Entropy","regularization","loss","weight",",","to","discourage","the","policy","distribution","from","being","``","too","certain","''","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","no","entropy","regularization",")",".","state_preprocessing","(","dict","[","specification","]",")",":","State","preprocessing","as","layer","or","list","of","layers",",","see","the","[","preprocessing","documentation","]","(","..","\/modules\/preprocessing.html",")",",","specified","per","state-type","or","-name","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","linear","normalization","of","bounded","float","states","to","[","-2.0",",","2.0","]",")",".","exploration","(","<","a","href=","''","..","\/modules\/parameters.html","''",">","parameter","<","\/a",">","|","dict","[","<","a","href=","''","..","\/modules\/parameters.html","''",">","parameter","<","\/a",">","]",",","float",">","=","0.0",")",":","Exploration",",","defined","as","the","probability","for","uniformly","random","output","in","case","of","`","bool","`","and","`","int","`","actions",",","and","the","standard","deviation","of","Gaussian","noise","added","to","every","output","in","case","of","`","float","`","actions",",","specified","globally","or","per","action-type","or","-name","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","no","exploration",")",".","variable_noise","(","<","a","href=","''","..","\/modules\/parameters.html","''",">","parameter","<","\/a",">",",","float",">","=","0.0",")",":","Add","Gaussian","noise","with","given","standard","deviation","to","all","trainable","variables",",","as","alternative","exploration","mechanism","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","no","variable","noise",")",".","parallel_interactions","(","int",">","0",")",":","Maximum","number","of","parallel","interactions","to","support",",","for","instance",",","to","enable","multiple","parallel","episodes",",","environments","or","agents","within","an","environment","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","1",")",".","config","(","specification",")",":","Additional","configuration","options",":","<","ul",">","<","li",">","<","b",">","name","<","\/b",">","(","<","i",">","string","<","\/i",">",")","&","ndash",";","Agent","name",",","used","e.g",".","for","TensorFlow","scopes","and","saver","default","filename","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","``","agent","''",")",".","<","li",">","<","b",">","device","<","\/b",">","(","<","i",">","string","<","\/i",">",")","&","ndash",";","Device","name","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","CPU",")",".","Different","from","(","un",")","supervised","deep","learning",",","RL","does","not","always","benefit","from","running","on","a","GPU",",","depending","on","environment","and","agent","configuration",".","In","particular","for","RL-typical","environments","with","low-dimensional","state","spaces","(","i.e.",",","no","images",")",",","one","usually","gets","better","performance","by","running","on","CPU","only",".","Consequently",",","Tensorforce","is","configured","to","run","on","CPU","by","default",",","which","can","be","changed",",","for","instance",",","by","setting","this","value","to","'GPU","'","instead",".","<","li",">","<","b",">","seed","<","\/b",">","(","<","i",">","int","<","\/i",">",")","&","ndash",";","Random","seed","to","set","for","Python",",","NumPy","(","both","set","globally","!",")","and","TensorFlow",",","environment","seed","may","have","to","be","set","separately","for","fully","deterministic","execution",",","generally","not","recommended","since","results","in","a","fully","deterministic","setting","are","less","meaningful\/representative","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","none",")",".","<","\/li",">","<","li",">","<","b",">","buffer_observe","<","\/b",">","(","<","i",">","false","|","``","episode","''","|","int",">","0","<","\/i",">",")","&","ndash",";","Number","of","timesteps","within","an","episode","to","buffer","before","calling","the","internal","observe","function",",","to","reduce","calls","to","TensorFlow","for","improved","performance","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","configuration-specific","maximum","number","which","can","be","buffered","without","affecting","performance",")",".","<","\/li",">","<","li",">","<","b",">","enable_int_action_masking","<","\/b",">","(","<","i",">","bool","<","\/i",">",")","&","ndash",";","Whether","int","action","options","can","be","masked","via","an","optional","``","[","ACTION-NAME","]","_mask","''","state","input","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","true",")",".","<","\/li",">","<","li",">","<","b",">","create_tf_assertions","<","\/b",">","(","<","i",">","bool","<","\/i",">",")","&","ndash",";","Whether","to","create","internal","TensorFlow","assertion","operations","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","true",")",".","<","\/li",">","<","li",">","<","b",">","eager_mode","<","\/b",">","(","<","i",">","bool","<","\/i",">",")","&","ndash",";","Whether","to","run","functions","eagerly","instead","of","running","as","a","traced","graph","function",",","can","be","helpful","for","debugging","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","false",")",".","<","\/li",">","<","li",">","<","b",">","tf_log_level","<","\/b",">","(","<","i",">","int",">","=","0","<","\/i",">",")","&","ndash",";","TensorFlow","log","level",",","additional","C++","logging","messages","can","be","enabled","by","setting","os.environ","[","``","TF_CPP_MIN_LOG_LEVEL","''","]","=","``","1","''","\/","''","2","''","before","importing","Tensorforce\/TensorFlow","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","40",",","only","error","and","critical",")",".","<","\/li",">","<","\/ul",">","saver","(","path","|","specification",")",":","TensorFlow","checkpoints","directory",",","or","checkpoint","manager","configuration","with","the","following","attributes",",","for","periodic","implicit","saving","as","alternative","to","explicit","saving","via","agent.save","(",")","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","no","saver",")",":","<","ul",">","<","li",">","<","b",">","directory","<","\/b",">","(","<","i",">","path","<","\/i",">",")","&","ndash",";","checkpoint","directory","(","<","span","style=","''","color",":","#","C00000","''",">","<","b",">","required","<","\/b",">","<","\/span",">",")",".","<","\/li",">","<","li",">","<","b",">","filename","<","\/b",">","(","<","i",">","string","<","\/i",">",")","&","ndash",";","checkpoint","filename","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","agent","name",")",".","<","\/li",">","<","li",">","<","b",">","frequency","<","\/b",">","(","<","i",">","int",">","0","<","\/i",">",")","&","ndash",";","how","frequently","to","save","a","checkpoint","(","<","span","style=","''","color",":","#","C00000","''",">","<","b",">","required","<","\/b",">","<","\/span",">",")",".","<","\/li",">","<","li",">","<","b",">","unit","<","\/b",">","(","<","i",">","''","timesteps","''","|","``","episodes","''","|","``","updates","''","<","\/i",">",")","&","ndash",";","frequency","unit","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","updates",")",".","<","\/li",">","<","li",">","<","b",">","max_checkpoints","<","\/b",">","(","<","i",">","int",">","0","<","\/i",">",")","&","ndash",";","maximum","number","of","checkpoints","to","keep","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","10",")",".","<","\/li",">","<","li",">","<","b",">","max_hour_frequency","<","\/b",">","(","<","i",">","int",">","0","<","\/i",">",")","&","ndash",";","ignoring","max-checkpoints",",","definitely","keep","a","checkpoint","in","given","hour","frequency","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","none",")",".","<","\/li",">","<","\/ul",">","summarizer","(","path","|","specification",")",":","TensorBoard","summaries","directory",",","or","summarizer","configuration","with","the","following","attributes","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","no","summarizer",")",":","<","ul",">","<","li",">","<","b",">","directory","<","\/b",">","(","<","i",">","path","<","\/i",">",")","&","ndash",";","summarizer","directory","(","<","span","style=","''","color",":","#","C00000","''",">","<","b",">","required","<","\/b",">","<","\/span",">",")",".","<","\/li",">","<","li",">","<","b",">","filename","<","\/b",">","(","<","i",">","path","<","\/i",">",")","&","ndash",";","summarizer","filename",",","max_summaries","does","not","apply","if","name","specified","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","``","summary-","%","Y","%","m","%","d-","%","H","%","M","%","S","''",")",".","<","\/li",">","<","li",">","<","b",">","max_summaries","<","\/b",">","(","<","i",">","int",">","0","<","\/i",">",")","&","ndash",";","maximum","number","of","(","generically-named",")","summaries","to","keep","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","7",",","number","of","different","colors","in","Tensorboard",")",".","<","\/li",">","<","li",">","<","b",">","flush","<","\/b",">","(","<","i",">","int",">","0","<","\/i",">",")","&","ndash",";","how","frequently","in","seconds","to","flush","the","summary","writer","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","10",")",".","<","\/li",">","<","li",">","<","b",">","summaries","<","\/b",">","(","<","i",">","''","all","''","|","iter","[","string","]","<","\/i",">",")","&","ndash",";","which","summaries","to","record",",","``","all","''","implies","all","numerical","summaries",",","so","all","summaries","except","``","graph","''","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","``","all","''",")",":","<","\/li",">","<","li",">","''","action-value","''",":","value","of","each","action","(","timestep-based",")","<","\/li",">","<","li",">","''","distribution","''",":","distribution","parameters","like","probabilities","or","mean","and","stddev","(","timestep-based",")","<","\/li",">","<","li",">","''","entropy","''",":","entropy","of","(","per-action",")","policy","distribution","(","s",")","(","timestep-based",")","<","\/li",">","<","li",">","''","graph","''",":","computation","graph","<","\/li",">","<","li",">","''","kl-divergence","''",":","KL-divergence","of","previous","and","updated","(","per-action",")","policy","distribution","(","s",")","(","update-based",")","<","\/li",">","<","li",">","''","loss","''",":","policy","and","baseline","loss","plus","loss","components","(","update-based",")","<","\/li",">","<","li",">","''","parameters","''",":","parameter","values","(","according","to","parameter","unit",")","<","\/li",">","<","li",">","''","reward","''",":","reward","per","timestep",",","episode","length","and","reward",",","plus","intermediate","reward\/return\/advantage","estimates","and","processed","values","(","timestep\/episode\/update-based",")","<","\/li",">","<","li",">","''","update-norm","''",":","global","norm","of","update","(","update-based",")","<","\/li",">","<","li",">","''","updates","''",":","mean","and","variance","of","update","tensors","per","variable","(","update-based",")","<","\/li",">","<","li",">","''","variables","''",":","mean","of","trainable","variables","tensors","(","update-based",")","<","\/li",">","<","\/ul",">","tracking","(","``","all","''","|","iter","[","string","]",")",":","Which","tensors","to","track",",","available","values","are","a","subset","of","the","values","of","summarizer","[","summaries","]","above","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","no","tracking",")",".","The","current","value","of","tracked","tensors","can","be","retrieved","via","tracked_tensors","(",")","at","any","time",",","however",",","note","that","tensor","values","change","at","different","timescales","(","timesteps",",","episodes",",","updates",")",".","recorder","(","path","|","specification",")",":","Traces","recordings","directory",",","or","recorder","configuration","with","the","following","attributes","(","see","[","record-and-pretrain","script","]","(","https",":","\/\/github.com\/tensorforce\/tensorforce\/blob\/master\/examples\/record_and_pretrain.py",")","for","example","application",")","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","no","recorder",")",":","<","ul",">","<","li",">","<","b",">","directory","<","\/b",">","(","<","i",">","path","<","\/i",">",")","&","ndash",";","recorder","directory","(","<","span","style=","''","color",":","#","C00000","''",">","<","b",">","required","<","\/b",">","<","\/span",">",")",".","<","\/li",">","<","li",">","<","b",">","frequency","<","\/b",">","(","<","i",">","int",">","0","<","\/i",">",")","&","ndash",";","how","frequently","in","episodes","to","record","traces","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","every","episode",")",".","<","\/li",">","<","li",">","<","b",">","start","<","\/b",">","(","<","i",">","int",">","=","0","<","\/i",">",")","&","ndash",";","how","many","episodes","to","skip","before","starting","to","record","traces","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","0",")",".","<","\/li",">","<","li",">","<","b",">","max-traces","<","\/b",">","(","<","i",">","int",">","0","<","\/i",">",")","&","ndash",";","maximum","number","of","traces","to","keep","(","<","span","style=","''","color",":","#","00C000","''",">","<","b",">","default","<","\/b",">","<","\/span",">",":","all",")",".","<","\/li",">"]
code: def __init__(
      # Required
      self,
      states,
      actions,
      update,
      optimizer,
      objective,
      reward_estimation,
      # Environment
      max_episode_timesteps=None,
      # Agent
      policy="auto",
      memory=None,
      # Baseline
      baseline=None,
      baseline_optimizer=None,
      baseline_objective=None,
      # Regularization
      l2_regularization=0.0,
      entropy_regularization=0.0,
      # Preprocessing
      state_preprocessing="linear_normalization",
      # Exploration
      exploration=0.0,
      variable_noise=0.0,
      # Parallel interactions
      parallel_interactions=1,
      # Config, saver, summarizer, tracking, recorder
      config=None,
      saver=None,
      summarizer=None,
      tracking=None,
      recorder=None,
      # Deprecated
      **kwargs,
  ):
      if "estimate_actions" in reward_estimation:
          raise TensorforceError.deprecated(
              name="Agent",
              argument="reward_estimation[estimate_actions]",
              replacement="reward_estimation[predict_action_values]",
          )
      if "estimate_terminal" in reward_estimation:
          raise TensorforceError.deprecated(
              name="Agent",
              argument="reward_estimation[estimate_terminal]",
              replacement="reward_estimation[predict_terminal_values]",
          )
      if summarizer is not None and "labels" in summarizer:
          raise TensorforceError.deprecated(
              name="Agent",
              argument="summarizer[labels]",
              replacement="summarizer[summaries]",
          )
      if "baseline_policy" in kwargs:
          raise TensorforceError.deprecated(
              name="Agent",
              argument="baseline_policy",
              replacement="baseline",
          )
      if "reward_preprocessing" in kwargs:
          raise TensorforceError.deprecated(
              name="Agent",
              argument="reward_preprocessing",
              replacement="reward_estimation[reward_processing]",
          )
      if "name" in kwargs:
          raise TensorforceError.deprecated(
              name="Agent", argument="name", replacement="config[name]"
          )
      if "buffer_observe" in kwargs:
          raise TensorforceError.deprecated(
              name="Agent",
              argument="buffer_observe",
              replacement="config[buffer_observe]",
          )
      if "device" in kwargs:
          raise TensorforceError.deprecated(
              name="Agent",
              argument="device",
              replacement="config[device]",
          )
      if "seed" in kwargs:
          raise TensorforceError.deprecated(
              name="Agent", argument="seed", replacement="config[seed]"
          )
      if len(kwargs) > 0:
          raise TensorforceError.invalid(
              name="Agent", argument=", ".join(kwargs)
          )
  
      if not hasattr(self, "spec"):
          self.spec = OrderedDict(
              agent="tensorforce",
              # Environment
              states=states,
              actions=actions,
              max_episode_timesteps=max_episode_timesteps,
              # Agent
              policy=policy,
              memory=memory,
              update=update,
              optimizer=optimizer,
              objective=objective,
              reward_estimation=reward_estimation,
              # Baseline
              baseline=baseline,
              baseline_optimizer=baseline_optimizer,
              baseline_objective=baseline_objective,
              # Regularization
              l2_regularization=l2_regularization,
              entropy_regularization=entropy_regularization,
              # Preprocessing
              state_preprocessing=state_preprocessing,
              # Exploration
              exploration=exploration,
              variable_noise=variable_noise,
              # Parallel interactions
              parallel_interactions=parallel_interactions,
              # Config, saver, summarizer, recorder
              config=config,
              saver=saver,
              summarizer=summarizer,
              tracking=tracking,
              recorder=recorder,
          )
  
      if memory is None:
          memory = dict(type="recent")
  
      if isinstance(update, int):
          update = dict(unit="timesteps", batch_size=update)
  
      if config is None:
          config = dict()
      else:
          config = dict(config)
  
      # TODO: should this change if summarizer is specified?
      if parallel_interactions > 1:
          if "buffer_observe" not in config:
              if max_episode_timesteps is None:
                  raise TensorforceError.required(
                      name="Agent",
                      argument="max_episode_timesteps",
                      condition="parallel_interactions > 1",
                  )
              config["buffer_observe"] = "episode"
          # elif config['buffer_observe'] < max_episode_timesteps:
          #     raise TensorforceError.value(
          #         name='Agent', argument='config[buffer_observe]',
          #         hint='< max_episode_timesteps', condition='parallel_interactions > 1'
          #     )
  
      elif update["unit"] == "timesteps":
          update_frequency = update.get(
              "frequency", update["batch_size"]
          )
          if "buffer_observe" not in config:
              if isinstance(update_frequency, int):
                  config["buffer_observe"] = update_frequency
              else:
                  config["buffer_observe"] = 1
          elif isinstance(update_frequency, int) and (
              config["buffer_observe"] == "episode"
              or config["buffer_observe"] > update_frequency
          ):
              raise TensorforceError.value(
                  name="Agent",
                  argument="config[buffer_observe]",
                  value=config["buffer_observe"],
                  hint="> update[frequency]",
                  condition='update[unit] = "timesteps"',
              )
  
      elif update["unit"] == "episodes":
          if "buffer_observe" not in config:
              config["buffer_observe"] = "episode"
  
      # reward_estimation = dict(reward_estimation)
      # if reward_estimation['horizon'] == 'episode':
      #     if max_episode_timesteps is None:
      #         raise TensorforceError.required(
      #             name='Agent', argument='max_episode_timesteps',
      #             condition='reward_estimation[horizon] = "episode"'
      #         )
      #     reward_estimation['horizon'] = max_episode_timesteps
  
      super().__init__(
          states=states,
          actions=actions,
          max_episode_timesteps=max_episode_timesteps,
          parallel_interactions=parallel_interactions,
          config=config,
          recorder=recorder,
      )
  
      self.model = TensorforceModel(
          states=self.states_spec,
          actions=self.actions_spec,
          max_episode_timesteps=self.max_episode_timesteps,
          policy=policy,
          memory=memory,
          update=update,
          optimizer=optimizer,
          objective=objective,
          reward_estimation=reward_estimation,
          baseline=baseline,
          baseline_optimizer=baseline_optimizer,
          baseline_objective=baseline_objective,
          l2_regularization=l2_regularization,
          entropy_regularization=entropy_regularization,
          state_preprocessing=state_preprocessing,
          exploration=exploration,
          variable_noise=variable_noise,
          parallel_interactions=self.parallel_interactions,
          config=self.config,
          saver=saver,
          summarizer=summarizer,
          tracking=tracking,
      )
  
file_path: tensorforce/tensorforce/agents/tensorforce.py
imports: from collections import OrderedDict
  import os
  from random import shuffle
  import numpy
  from tensorforce import TensorforceError, util
  from tensorforce.agents import Agent
  from tensorforce.core import ArrayDict
  from tensorforce.core.models import TensorforceModel
label_desc: Use image node_id 1 to create a new TensorforceAgent object from inherited base classes: Agent with example: obj = TensorforceAgent(states, actions, update, optimizer, objective, reward_estimation, max_episode_timesteps, policy, memory, baseline, baseline_optimizer, baseline_objective, l2_regularization, entropy_regularization, state_preprocessing, exploration, variable_noise, parallel_interactions, config, saver, summarizer, tracking, recorder)