class_docstr: Implements the Wheel Bandit environment.
  
  This environment implements the wheel bandit from Section 5.4 of [1] (please
  see references below) as a subclass of BanditPyEnvironment.
  
  Context features are sampled uniformly at random in the unit circle in R^2.
  There are 5 possible actions. There exists an exploration parameter `delta`
  in (0, 1) that determines the difficulty of the problem and the need for
  exploration.
  
  References:
  [1]. Carlos Riquelme, George Tucker, Jasper Snoek
  "Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep
  Networks for Thompson Sampling", International Conference on Learning
  Representations (ICLR) 2018.
  https://arxiv.org/abs/1802.09127
class_docstr_tok: ["Implements","the","Wheel","Bandit","environment",".","This","environment","implements","the","wheel","bandit","from","Section","5.4","of","[","1","]","(","please","see","references","below",")","as","a","subclass","of","BanditPyEnvironment",".","Context","features","are","sampled","uniformly","at","random","in","the","unit","circle","in","R^2",".","There","are","5","possible","actions",".","There","exists","an","exploration","parameter","`","delta","`","in","(","0",",","1",")","that","determines","the","difficulty","of","the","problem","and","the","need","for","exploration",".","References",":","[","1","]",".","Carlos","Riquelme",",","George","Tucker",",","Jasper","Snoek","''","Deep","Bayesian","Bandits","Showdown",":","An","Empirical","Comparison","of","Bayesian","Deep","Networks","for","Thompson","Sampling","''",",","International","Conference","on","Learning","Representations","(","ICLR",")","2018.","https",":","\/\/arxiv.org\/abs\/1802.09127"]
code: def _reward_fn(self, observation, action):
      # Sample rewards for all actions.
      r_all = np.random.normal(
          self._mu_base,
          self._std_base,
          size=(self._batch_size, _NUM_ACTIONS),
      )
  
      # Compute the reward inside.
      row_norms = np.linalg.norm(observation, ord=2, axis=1)
      is_norm_below_delta = np.float32(row_norms <= self._delta)
      reward_inside = (
          is_norm_below_delta
          * r_all[np.arange(self._batch_size), action]
      )
  
      # Compute the reward outside.
      high_reward = np.random.normal(
          self._mu_high, self._std_high, size=(self._batch_size)
      )
      signs = np.sign(observation)
      optimal_actions = [_SIGNS_TO_OPT_ACTION[tuple(x)] for x in signs]
      r_outside = r_all
      r_outside[
          np.arange(self._batch_size), optimal_actions
      ] = high_reward
  
      reward_outside = (1.0 - is_norm_below_delta) * r_outside[
          np.arange(self._batch_size), action
      ]
  
      reward_final = reward_inside + reward_outside
      return reward_final
  
file_path: agents/tf_agents/bandits/environments/wheel_py_environment.py
imports: from __future__ import absolute_import
  from typing import Optional, Sequence, Text
  import gin
  import numpy
  from tf_agents.bandits.environments import bandit_py_environment
  from tf_agents.specs import array_spec
  from tf_agents.trajectories import time_step
  from tf_agents.typing import types
label_desc: Use image node_id 4 for calling the WheelPyEnvironment obj's underlying member method code with example usage: obj._reward_fn(observation, action) and returns: reward_final