class_docstr: A Thompson Sampling agent for non-contextual Bernoulli bandit problems.
  
  In Bernoulli bandit problems, the reward for each arm is a Bernoulli
  distribution with parameter p. The agent assumes that the prior distribution
  is beta distributed, which is conjugate to the bernoulli distribution and
  the posterior distribution of each arm admits simple incremental updates.
  For a reference, see e.g., Chapter 3 in "A Tutorial on Thompson Sampling" by
  Russo et al. (https://web.stanford.edu/~bvr/pubs/TS_Tutorial.pdf)
class_docstr_tok: ["A","Thompson","Sampling","agent","for","non-contextual","Bernoulli","bandit","problems",".","In","Bernoulli","bandit","problems",",","the","reward","for","each","arm","is","a","Bernoulli","distribution","with","parameter","p.","The","agent","assumes","that","the","prior","distribution","is","beta","distributed",",","which","is","conjugate","to","the","bernoulli","distribution","and","the","posterior","distribution","of","each","arm","admits","simple","incremental","updates",".","For","a","reference",",","see","e.g.",",","Chapter","3","in","``","A","Tutorial","on","Thompson","Sampling","''","by","Russo","et","al",".","(","https",":","\/\/web.stanford.edu\/~bvr\/pubs\/TS_Tutorial.pdf",")"]
docstr: Creates a Bernoulli Thompson Sampling Agent.
  
  Args:
    time_step_spec: A `TimeStep` spec of the expected time_steps.
    action_spec: A nest of `BoundedTensorSpec` representing the actions.
    variable_collection: Instance of `BernoulliBanditVariableCollection`.
      Collection of variables to be updated by the agent. If `None`, a new
      instance of `BernoulliBanditVariableCollection` will be created.
    dtype: The type of the variables. Should be one of `tf.float32` or
      `tf.float64`.
    batch_size: optional int with the batch size. It defaults to 1.
    observation_and_action_constraint_splitter: A function used for masking
      valid/invalid actions with each state of the environment. The function
      takes in a full observation and returns a tuple consisting of 1) the
      part of the observation intended as input to the bandit agent and
      policy, and 2) the boolean mask. This function should also work with a
      `TensorSpec` as input, and should output `TensorSpec` objects for the
      observation and mask.
    emit_policy_info: (tuple of strings) what side information we want to get
      as part of the policy info. Allowed values can be found in
      `policy_utilities.PolicyInfo`.
    name: Python str name of this agent. All variables in this module will
      fall under that name. Defaults to the class name.
  
  Raises:
    ValueError: If the action spec contains more than one action or it is
      not a bounded scalar int32 spec with minimum 0.
    TypeError: if variable_collection is not an instance of
      `BernoulliBanditVariableCollection`.
docstr_tok: ["Creates","a","Bernoulli","Thompson","Sampling","Agent",".","Args",":","time_step_spec",":","A","`","TimeStep","`","spec","of","the","expected","time_steps",".","action_spec",":","A","nest","of","`","BoundedTensorSpec","`","representing","the","actions",".","variable_collection",":","Instance","of","`","BernoulliBanditVariableCollection","`",".","Collection","of","variables","to","be","updated","by","the","agent",".","If","`","None","`",",","a","new","instance","of","`","BernoulliBanditVariableCollection","`","will","be","created",".","dtype",":","The","type","of","the","variables",".","Should","be","one","of","`","tf.float32","`","or","`","tf.float64","`",".","batch_size",":","optional","int","with","the","batch","size",".","It","defaults","to","1.","observation_and_action_constraint_splitter",":","A","function","used","for","masking","valid\/invalid","actions","with","each","state","of","the","environment",".","The","function","takes","in","a","full","observation","and","returns","a","tuple","consisting","of","1",")","the","part","of","the","observation","intended","as","input","to","the","bandit","agent","and","policy",",","and","2",")","the","boolean","mask",".","This","function","should","also","work","with","a","`","TensorSpec","`","as","input",",","and","should","output","`","TensorSpec","`","objects","for","the","observation","and","mask",".","emit_policy_info",":","(","tuple","of","strings",")","what","side","information","we","want","to","get","as","part","of","the","policy","info",".","Allowed","values","can","be","found","in","`","policy_utilities.PolicyInfo","`",".","name",":","Python","str","name","of","this","agent",".","All","variables","in","this","module","will","fall","under","that","name",".","Defaults","to","the","class","name",".","Raises",":","ValueError",":","If","the","action","spec","contains","more","than","one","action","or","it","is","not","a","bounded","scalar","int32","spec","with","minimum","0",".","TypeError",":","if","variable_collection","is","not","an","instance","of","`","BernoulliBanditVariableCollection","`","."]
code: def __init__(
      self,
      time_step_spec: types.TimeStep,
      action_spec: types.BoundedTensorSpec,
      variable_collection: Optional[
          BernoulliBanditVariableCollection
      ] = None,
      dtype: tf.DType = tf.float32,
      batch_size: Optional[int] = 1,
      observation_and_action_constraint_splitter: Optional[
          types.Splitter
      ] = None,
      emit_policy_info: Sequence[Text] = (),
      name: Optional[Text] = None,
  ):
      """Creates a Bernoulli Thompson Sampling Agent.
  
      Args:
        time_step_spec: A `TimeStep` spec of the expected time_steps.
        action_spec: A nest of `BoundedTensorSpec` representing the actions.
        variable_collection: Instance of `BernoulliBanditVariableCollection`.
          Collection of variables to be updated by the agent. If `None`, a new
          instance of `BernoulliBanditVariableCollection` will be created.
        dtype: The type of the variables. Should be one of `tf.float32` or
          `tf.float64`.
        batch_size: optional int with the batch size. It defaults to 1.
        observation_and_action_constraint_splitter: A function used for masking
          valid/invalid actions with each state of the environment. The function
          takes in a full observation and returns a tuple consisting of 1) the
          part of the observation intended as input to the bandit agent and
          policy, and 2) the boolean mask. This function should also work with a
          `TensorSpec` as input, and should output `TensorSpec` objects for the
          observation and mask.
        emit_policy_info: (tuple of strings) what side information we want to get
          as part of the policy info. Allowed values can be found in
          `policy_utilities.PolicyInfo`.
        name: Python str name of this agent. All variables in this module will
          fall under that name. Defaults to the class name.
  
      Raises:
        ValueError: If the action spec contains more than one action or it is
          not a bounded scalar int32 spec with minimum 0.
        TypeError: if variable_collection is not an instance of
          `BernoulliBanditVariableCollection`.
      """
      tf.Module.__init__(self, name=name)
      common.tf_agents_gauge.get_cell("TFABandit").set(True)
      self._observation_and_action_constraint_splitter = (
          observation_and_action_constraint_splitter
      )
      self._num_actions = (
          policy_utilities.get_num_actions_from_tensor_spec(action_spec)
      )
  
      self._dtype = dtype
      if variable_collection is None:
          variable_collection = BernoulliBanditVariableCollection(
              num_actions=self._num_actions, dtype=dtype
          )
      elif not isinstance(
          variable_collection, BernoulliBanditVariableCollection
      ):
          raise TypeError(
              "Parameter `variable_collection` should be "
              "of type `BernoulliBanditVariableCollection`."
          )
      self._variable_collection = variable_collection
      self._alpha = variable_collection.alpha
      self._beta = variable_collection.beta
      self._batch_size = batch_size
      policy = bernoulli_policy.BernoulliThompsonSamplingPolicy(
          time_step_spec,
          action_spec,
          self._alpha,
          self._beta,
          observation_and_action_constraint_splitter,
          emit_policy_info=emit_policy_info,
      )
  
      super(BernoulliThompsonSamplingAgent, self).__init__(
          time_step_spec,
          action_spec,
          policy,
          collect_policy=policy,
          train_sequence_length=None,
      )
      self._as_trajectory = data_converter.AsTrajectory(
          self.data_context, sequence_length=None
      )
  
file_path: agents/tf_agents/bandits/agents/bernoulli_thompson_sampling_agent.py
imports: from typing import Optional, Sequence, Text
  import gin
  import tensorflow
  from tf_agents.agents import data_converter
  from tf_agents.agents import tf_agent
  from tf_agents.bandits.policies import bernoulli_thompson_sampling_policy
  from tf_agents.policies import utils
  from tf_agents.typing import types
  from tf_agents.utils import common
  from tf_agents.utils import nest_utils
label_desc: Use image node_id 1 to create a new BernoulliThompsonSamplingAgent object from inherited base classes: tf_agent with example: obj = BernoulliThompsonSamplingAgent(time_step_spec, action_spec, variable_collection, dtype, batch_size, observation_and_action_constraint_splitter, emit_policy_info, name)