class_docstr: Thompson sampling.
  
  Thompson sampling is often used with a Beta distribution. However, any probability distribution
  can be used, as long it makes sense with the reward shape. For instance, a Beta distribution
  is meant to be used with binary rewards, while a Gaussian distribution is meant to be used with
  continuous rewards.
  
  The randomness of a distribution is controlled by its seed. The seed should not set within the
  distribution, but should rather be defined in the policy parametrization. In other words, you
  should do this:
  
  ```python
  
  policy = ThompsonSampling(dist=proba.Beta(1, 1), seed=42)
  
  ```
  
  and not this:
  
  ```python
  
  policy = ThompsonSampling(dist=proba.Beta(1, 1, seed=42))
  
  ```
  
  Parameters
  ----------
  reward_obj
      A distribution to sample from.
  burn_in
      The number of steps to use for the burn-in phase. Each arm is given the chance to be pulled
      during the burn-in phase. This is useful to mitigate selection bias.
  seed
      Random number generator seed for reproducibility.
  
  Examples
  --------
  
  >>> import gym
  >>> from river import bandit
  >>> from river import proba
  >>> from river import stats
  
  >>> env = gym.make(
  ...     'river_bandits/CandyCaneContest-v0'
  ... )
  >>> _ = env.reset(seed=42)
  >>> _ = env.action_space.seed(123)
  
  >>> policy = bandit.ThompsonSampling(reward_obj=proba.Beta(), seed=101)
  
  >>> metric = stats.Sum()
  >>> while True:
  ...     arm = policy.pull(range(env.action_space.n))
  ...     observation, reward, terminated, truncated, info = env.step(arm)
  ...     policy.update(arm, reward)
  ...     metric.update(reward)
  ...     if terminated or truncated:
  ...         break
  
  >>> metric
  Sum: 820.
  
  References
  ----------
  [^1]: [An Empirical Evaluation of Thompson Sampling](https://proceedings.neurips.cc/paper/2011/file/e53a0a2978c28872a4505bdb51db06dc-Paper.pdf)
code: def _pull(self, arm_ids):
      return max(
          arm_ids, key=lambda arm_id: self._rewards[arm_id].sample()
      )
  
file_path: river/river/bandit/thompson.py
imports: from __future__ import annotations
  import random
  from river import bandit, proba
label_desc: Use image node_id 3 for calling the ThompsonSampling obj's underlying member method code with example usage: obj._pull(arm_ids) and returns: max