code: def evaluate_offline(
      policy: bandit.base.Policy,
      history: History | bandit.datasets.BanditDataset,
      reward_stat: stats.base.Univariate = None,
  ) -> tuple[stats.base.Univariate, int]:
      """Evaluate a policy on historical logs using replay.
  
      This is a high-level utility function for evaluating a policy using the replay methodology.
      This methodology is an off-policy evaluation method. It does not require an environment, and is
      instead data-driven.
  
      At each step, an arm is pulled from the provided policy. If the arm is the same as the arm
      that was pulled in the historical data, the reward is used to update the policy. If the arm is
      different, the reward is ignored. This is the off-policy aspect of the evaluation.
  
      Parameters
      ----------
      policy
          The policy to evaluate.
      history
          The history of the bandit problem. This is a generator that yields tuples of the form
          `(arms, context, arm, reward)`.
      reward_stat
          The reward statistic to use. Defaults to `stats.Sum`.
  
      Returns
      -------
      The reward statistic, along with the number of times the policy pulled the same arm as the
      historical data.
  
      Examples
      --------
  
      >>> import random
      >>> from river import bandit
  
      >>> rng = random.Random(42)
      >>> arms = ['A', 'B', 'C']
      >>> clicks = [
      ...     (
      ...         arms,
      ...         # no context
      ...         None,
      ...         # random arm
      ...         rng.choice(arms),
      ...         # reward
      ...         rng.random() > 0.5
      ...     )
      ...     for _ in range(1000)
      ... ]
  
      >>> total_reward, n_samples_used = bandit.evaluate_offline(
      ...     policy=bandit.EpsilonGreedy(0.1, seed=42),
      ...     history=clicks,
      ... )
  
      >>> total_reward
      Sum: 172.
  
      >>> n_samples_used
      321
  
      This also works out of the box with datasets that inherit from `river.bandit.BanditDataset`.
  
      >>> news = bandit.datasets.NewsArticles()
      >>> total_reward, n_samples_used = bandit.evaluate_offline(
      ...     policy=bandit.RandomPolicy(seed=42),
      ...     history=news,
      ... )
  
      >>> total_reward, n_samples_used
      (Sum: 105., 1027)
  
      As expected, the policy succeeds in roughly 10% of cases. Indeed, there are 10 arms and 10000
      samples, so the expected number of successes is 1000.
  
      References
      ----------
      [^1]: [Offline Evaluation of Multi-Armed Bandit Algorithms in Python using Replay](https://jamesrledoux.com/algorithms/offline-bandit-evaluation/)
      [^2]: [Unbiased Offline Evaluation of Contextual-bandit-based News Article Recommendation Algorithms](https://arxiv.org/pdf/1003.5956.pdf)
      [^3]: [Understanding Inverse Propensity Score for Contextual Bandits](https://chanind.github.io/ai/2022/03/14/inverse-propensity-score-contextual-bandits.html)
  
      """
  
      if isinstance(history, bandit.datasets.BanditDataset):
          arms = history.arms
          history = (
              (arms, context, arm, reward)
              for context, arm, reward in history
          )
  
      reward_stat = reward_stat or stats.Sum()
  
      is_contextual = isinstance(policy, bandit.base.ContextualPolicy)
  
      for arms_available, context, chosen_arm, reward in history:
          probability = 1  # TODO: use inverse propensity scoring
          arm = (
              policy.pull(arms_available, context=context)  # type: ignore[call-arg]
              if is_contextual
              else policy.pull(arms_available)
          )
  
          # Do nothing if the chosen arm differs from the arm in the historical data
          if chosen_arm != arm:
              continue
  
          if is_contextual:
              policy.update(arm, context, reward)
          else:
              policy.update(arm, reward)
          reward_stat.update(reward / probability)  # type: ignore
  
      # Normalize the reward statistic by the number of times the policy pulled the same arm as what
      # was pulled in the historical data
      return reward_stat, policy._n
  
file_path: river/river/bandit/evaluate.py
imports: from __future__ import annotations
  import copy
  import random
  import typing
  from river import bandit, stats
  from .base import ArmID
label_desc: Use image node_id 2 for calling a global function with example usage: evaluate_offline(policy, history, reward_stat) and returns: reward_stat,  policy