class_docstr: The Doubly Robust estimator.
  
  Let s_t, a_t, and r_t be the state, action, and reward at timestep t.
  
  This method trains a Q-model for the evaluation policy \pi_e on behavior
  data generated by \pi_b. Currently, RLlib implements this using
  Fitted-Q Evaluation (FQE). You can also implement your own model
  and pass it in as `q_model_config = {"type": your_model_class, **your_kwargs}`.
  
  For behavior policy \pi_b and evaluation policy \pi_e, define the
  cumulative importance ratio at timestep t as:
  p_t = \sum_{t'=0}^t (\pi_e(a_{t'} | s_{t'}) / \pi_b(a_{t'} | s_{t'})).
  
  Consider an episode with length T. Let V_T = 0.
  For all t in {0, T - 1}, use the following recursive update:
  V_t^DR = (\sum_{a \in A} \pi_e(a | s_t) Q(s_t, a))
      + p_t * (r_t + \gamma * V_{t+1}^DR - Q(s_t, a_t))
  
  This estimator computes the expected return for \pi_e for an episode as:
  V^{\pi_e}(s_0) = V_0^DR
  and returns the mean and standard deviation over episodes.
  
  For more information refer to https://arxiv.org/pdf/1911.06854.pdf
code: def estimate_on_single_episode(
      self, episode: SampleBatch
  ) -> Dict[str, Any]:
      estimates_per_epsiode = {}
  
      rewards, old_prob = episode["rewards"], episode["action_prob"]
      new_prob = self.compute_action_probs(episode)
  
      weight = new_prob / old_prob
  
      v_behavior = 0.0
      v_target = 0.0
      q_values = self.model.estimate_q(episode)
      q_values = convert_to_numpy(q_values)
      v_values = self.model.estimate_v(episode)
      v_values = convert_to_numpy(v_values)
      assert q_values.shape == v_values.shape == (episode.count,)
  
      for t in reversed(range(episode.count)):
          v_behavior = rewards[t] + self.gamma * v_behavior
          v_target = v_values[t] + weight[t] * (
              rewards[t] + self.gamma * v_target - q_values[t]
          )
      v_target = v_target.item()
  
      estimates_per_epsiode["v_behavior"] = v_behavior
      estimates_per_epsiode["v_target"] = v_target
  
      return estimates_per_epsiode
  
imports: import logging
  import numpy
  import math
  import pandas
  from typing import Dict, Any, Optional, List
  from ray.data import Dataset
  from ray.rllib.policy import Policy
  from ray.rllib.policy.sample_batch import SampleBatch, convert_ma_batch_to_sample_batch
  from ray.rllib.utils.annotations import DeveloperAPI, override
  from ray.rllib.utils.typing import SampleBatchType
  from ray.rllib.utils.numpy import convert_to_numpy
  from ray.rllib.offline.estimators.off_policy_estimator import OffPolicyEstimator
  from ray.rllib.offline.estimators.fqe_torch_model import FQETorchModel
  from ray.rllib.offline.offline_evaluator import OfflineEvaluator
  from ray.rllib.offline.offline_evaluation_utils import compute_is_weights, compute_q_and_v_values
label_desc: Use image node_id 2 for calling the DoublyRobust obj's underlying member method code with example usage: obj.estimate_on_single_episode(episode) and returns: estimates_per_epsiode