class_docstr: An agent that mixes a set of agents and updates the weights with Exp3.
  
  For a reference on EXP3, see `Bandit Algorithms` by Tor Lattimore and Csaba
  Szepesvari (https://tor-lattimore.com/downloads/book/book.pdf).
  
  The update uses a slighlty modified version of EXP3 to make sure that the
  weights do not go to one seemingly good agent in the very beginning. To smooth
  the weights, two extra measures are taken:
  
  1. A forgetting factor makes sure that the aggregated reward estimates do not
  grow indefinitely.
  2. The `inverse temperature` has a maximum parameter that prevents it from
  growing indefinitely.
  
  It is generally a good idea to set
  
  ```
  forgetting_factor = 1 - (1 / max_inverse_temperature)
  ```
  
  so that the two smoothing factors work together nicely.
  
  For every data sample, the agent updates the sub-agent that was used to make
  the action choice in that sample. For this update to happen, the mixture agent
  needs to have the information on which sub-agent is "responsible" for the
  action. This information is in a policy info field `mixture_choice_info`.
docstr: Initializes an instance of `Exp3MixtureAgent`.
  
  Args:
    agents: List of TF-Agents agents that this mixture agent trains.
    variable_collection: An instance of `Exp3VariableCollection`. If not set,
      A default one will be created. It contains all the variables that are
      needed to restore the mixture agent, excluding the variables of the
      subagents.
    forgetting: A float value in (0, 1]. This is how much the estimated reward
      aggregates are shrinked in every training step.
    max_inverse_temperature: This value caps the inverse temperature that
      would otherwise grow as the square root of the number of samples seen.
    name: Name fo this instance of `Exp3MixtureAgent`.
docstr_tok: ["Initializes","an","instance","of","`","Exp3MixtureAgent","`",".","Args",":","agents",":","List","of","TF-Agents","agents","that","this","mixture","agent","trains",".","variable_collection",":","An","instance","of","`","Exp3VariableCollection","`",".","If","not","set",",","A","default","one","will","be","created",".","It","contains","all","the","variables","that","are","needed","to","restore","the","mixture","agent",",","excluding","the","variables","of","the","subagents",".","forgetting",":","A","float","value","in","(","0",",","1","]",".","This","is","how","much","the","estimated","reward","aggregates","are","shrinked","in","every","training","step",".","max_inverse_temperature",":","This","value","caps","the","inverse","temperature","that","would","otherwise","grow","as","the","square","root","of","the","number","of","samples","seen",".","name",":","Name","fo","this","instance","of","`","Exp3MixtureAgent","`","."]
code: def __init__(
      self,
      agents: List[tf_agent.TFAgent],
      variable_collection: Optional[
          Exp3MixtureVariableCollection
      ] = None,
      forgetting: float = 0.999,
      max_inverse_temperature: float = 1000.0,
      name: Optional[Text] = None,
  ):
      """Initializes an instance of `Exp3MixtureAgent`.
  
      Args:
        agents: List of TF-Agents agents that this mixture agent trains.
        variable_collection: An instance of `Exp3VariableCollection`. If not set,
          A default one will be created. It contains all the variables that are
          needed to restore the mixture agent, excluding the variables of the
          subagents.
        forgetting: A float value in (0, 1]. This is how much the estimated reward
          aggregates are shrinked in every training step.
        max_inverse_temperature: This value caps the inverse temperature that
          would otherwise grow as the square root of the number of samples seen.
        name: Name fo this instance of `Exp3MixtureAgent`.
      """
      self._num_agents = len(agents)
      self._forgetting = forgetting
      self._max_inverse_temperature = max_inverse_temperature
      if variable_collection is None:
          variable_collection = Exp3MixtureVariableCollection(
              self._num_agents
          )
      elif not isinstance(
          variable_collection, Exp3MixtureVariableCollection
      ):
          raise TypeError(
              "Parameter `variable_collection` should be "
              "of type `MixtureVariableCollection`."
          )
      elif (
          variable_collection.reward_aggregates.shape
          != self._num_agents
      ):
          raise ValueError(
              "`variable_collection.reward_aggregates` should have "
              "shape `[len(agents)]`."
          )
      self._variable_collection = variable_collection
  
      # The `_mixture_weights` value is reassigned in every training step and only
      # depends on reward aggregates and inverse temperature. This variable is not
      # part of the variable collection because it is not needed to restore an
      # agent. The only reason why this value is a tf.Variable is because this way
      # the categorical distribution is dynamically parameterized.
  
      self._mixture_weights = tf.Variable(
          tf.zeros_like(variable_collection.reward_aggregates)
      )
      mixture_distribution = tfd.Categorical(
          logits=self._mixture_weights
      )
      super(Exp3MixtureAgent, self).__init__(
          mixture_distribution, agents, name=name
      )
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  from typing import List, Optional, Text
  import gin
  import tensorflow
  import tensorflow_probability
  from tf_agents.agents import tf_agent
  from tf_agents.bandits.agents import mixture_agent
  from tf_agents.bandits.policies import mixture_policy
  from tf_agents.utils import nest_utils
label_desc: Use image node_id 1 to create a new Exp3MixtureAgent object from inherited base classes: mixture_agent with example: obj = Exp3MixtureAgent(agents, variable_collection, forgetting, max_inverse_temperature, name)