class_docstr: Implements the Wheel Bandit environment.
  
  This environment implements the wheel bandit from Section 5.4 of [1] (please
  see references below) as a subclass of BanditPyEnvironment.
  
  Context features are sampled uniformly at random in the unit circle in R^2.
  There are 5 possible actions. There exists an exploration parameter `delta`
  in (0, 1) that determines the difficulty of the problem and the need for
  exploration.
  
  References:
  [1]. Carlos Riquelme, George Tucker, Jasper Snoek
  "Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep
  Networks for Thompson Sampling", International Conference on Learning
  Representations (ICLR) 2018.
  https://arxiv.org/abs/1802.09127
class_docstr_tok: ["Implements","the","Wheel","Bandit","environment",".","This","environment","implements","the","wheel","bandit","from","Section","5.4","of","[","1","]","(","please","see","references","below",")","as","a","subclass","of","BanditPyEnvironment",".","Context","features","are","sampled","uniformly","at","random","in","the","unit","circle","in","R^2",".","There","are","5","possible","actions",".","There","exists","an","exploration","parameter","`","delta","`","in","(","0",",","1",")","that","determines","the","difficulty","of","the","problem","and","the","need","for","exploration",".","References",":","[","1","]",".","Carlos","Riquelme",",","George","Tucker",",","Jasper","Snoek","''","Deep","Bayesian","Bandits","Showdown",":","An","Empirical","Comparison","of","Bayesian","Deep","Networks","for","Thompson","Sampling","''",",","International","Conference","on","Learning","Representations","(","ICLR",")","2018.","https",":","\/\/arxiv.org\/abs\/1802.09127"]
docstr: Initializes the Wheel Bandit environment.
  
  Args:
    delta: float in `(0, 1)`. Exploration parameter.
    mu_base: (vector of float) Mean reward for each action, if the context
      norm is below delta. The size of the vector is expected to be 5 (i.e.,
      equal to the number of actions.)
    std_base: (vector of float) std of the Gaussian reward for each action if
      the context norm is below delta. The size of the vector is expected to
      be 5 (i.e., equal to the number of actions.)
    mu_high: (float) Mean reward for the optimal action if the context norm is
      above delta.
    std_high: (float) Reward std for optimal action if the context norm is
      above delta.
    batch_size: (optional) (int) Number of observations generated per call.
    name: (optional) The name of this environment instance.
docstr_tok: ["Initializes","the","Wheel","Bandit","environment",".","Args",":","delta",":","float","in","`","(","0",",","1",")","`",".","Exploration","parameter",".","mu_base",":","(","vector","of","float",")","Mean","reward","for","each","action",",","if","the","context","norm","is","below","delta",".","The","size","of","the","vector","is","expected","to","be","5","(","i.e.",",","equal","to","the","number","of","actions",".",")","std_base",":","(","vector","of","float",")","std","of","the","Gaussian","reward","for","each","action","if","the","context","norm","is","below","delta",".","The","size","of","the","vector","is","expected","to","be","5","(","i.e.",",","equal","to","the","number","of","actions",".",")","mu_high",":","(","float",")","Mean","reward","for","the","optimal","action","if","the","context","norm","is","above","delta",".","std_high",":","(","float",")","Reward","std","for","optimal","action","if","the","context","norm","is","above","delta",".","batch_size",":","(","optional",")","(","int",")","Number","of","observations","generated","per","call",".","name",":","(","optional",")","The","name","of","this","environment","instance","."]
code: def __init__(
      self,
      delta: float,
      mu_base: Sequence[float],
      std_base: Sequence[float],
      mu_high: float,
      std_high: float,
      batch_size: Optional[int] = None,
      name: Optional[Text] = "wheel",
  ):
      """Initializes the Wheel Bandit environment.
  
      Args:
        delta: float in `(0, 1)`. Exploration parameter.
        mu_base: (vector of float) Mean reward for each action, if the context
          norm is below delta. The size of the vector is expected to be 5 (i.e.,
          equal to the number of actions.)
        std_base: (vector of float) std of the Gaussian reward for each action if
          the context norm is below delta. The size of the vector is expected to
          be 5 (i.e., equal to the number of actions.)
        mu_high: (float) Mean reward for the optimal action if the context norm is
          above delta.
        std_high: (float) Reward std for optimal action if the context norm is
          above delta.
        batch_size: (optional) (int) Number of observations generated per call.
        name: (optional) The name of this environment instance.
      """
      self._batch_size = 1 if batch_size is None else batch_size
      if delta <= 0 or delta >= 1:
          raise ValueError(
              "Delta must be in (0, 1), but saw delta: %g" % (delta,)
          )
      self._delta = delta
      self._mu_base = np.asarray(mu_base, dtype=np.float32)
      if self._mu_base.shape != (5,):
          raise ValueError(
              "The length of 'mu_base' must be 5, but saw 'mu_base': %s"
              % (self._mu_base,)
          )
      self._std_base = np.asarray(std_base, dtype=np.float32)
      if self._std_base.shape != (5,):
          raise ValueError("The length of 'std_base' must be 5.")
  
      self._mu_high = mu_high
      self._std_high = std_high
  
      # The first action should have higher mean reward that the other actions.
      if self._mu_base[0] != max(self._mu_base):
          raise ValueError(
              "The first action in mu_base should have the highest "
              "reward; got {}".format(self._mu_base)
          )
  
      action_spec = array_spec.BoundedArraySpec(
          shape=(),
          dtype=np.int32,
          minimum=0,
          maximum=_NUM_ACTIONS - 1,
          name="action",
      )
      observation_spec = array_spec.ArraySpec(
          shape=(_CONTEXT_DIM,), dtype=np.float32, name="observation"
      )
      self._time_step_spec = ts.time_step_spec(observation_spec)
      self._observation = np.zeros((self._batch_size, _CONTEXT_DIM))
      super(WheelPyEnvironment, self).__init__(
          observation_spec, action_spec, name=name
      )
  
file_path: agents/tf_agents/bandits/environments/wheel_py_environment.py
imports: from __future__ import absolute_import
  from typing import Optional, Sequence, Text
  import gin
  import numpy
  from tf_agents.bandits.environments import bandit_py_environment
  from tf_agents.specs import array_spec
  from tf_agents.trajectories import time_step
  from tf_agents.typing import types
label_desc: Use image node_id 1 to create a new WheelPyEnvironment object from inherited base classes: bandit_py_environment with example: obj = WheelPyEnvironment(delta, mu_base, std_base, mu_high, std_high, batch_size, name)