class_docstr: A batch RL graph manager creates a scenario of learning from a dataset without a simulator.
  
  If an environment is given (useful either for research purposes, or for experimenting with a toy problem before
  actually working with a real dataset), we can use it in order to collect a dataset to later be used to train the
  actual agent. The collected dataset, in this case, can be collected either by randomly acting in the environment
  (only running in heatup), or alternatively by training a different agent in the environment and using its collected
  data as a dataset. If an experience generating agent parameters are given, we will instantiate this agent and use it
   in order to train on the environment and then use this dataset to actually train an agent. Otherwise, we will
   collect a random dataset.
  :param agent_params: the parameters of the agent to train using batch RL
  :param env_params: [optional] environment parameters, for cases where we want to first collect a dataset
  :param vis_params: visualization parameters
  :param preset_validation_params: preset validation parameters, to be used for testing purposes
  :param name: graph name
  :param spaces_definition: when working with a dataset, we need to get a description of the actual state and action
                            spaces of the problem
  :param reward_model_num_epochs: the number of epochs to go over the dataset for training a reward model for the
                                  'direct method' and 'doubly robust' OPE methods.
  :param train_to_eval_ratio: percentage of the data transitions to be used for training vs. evaluation. i.e. a value
                              of 0.8 means ~80% of the transitions will be used for training and ~20% will be used for
                               evaluation using OPE.
  :param experience_generating_agent_params: [optional] parameters of an agent to be trained vs. an environment, whose
                                             his collected experience will be used to train the acutal (another) agent
  :param experience_generating_schedule_params: [optional] graph scheduling parameters for training the experience
                                                generating agent
class_docstr_tok: ["A","batch","RL","graph","manager","creates","a","scenario","of","learning","from","a","dataset","without","a","simulator",".","If","an","environment","is","given","(","useful","either","for","research","purposes",",","or","for","experimenting","with","a","toy","problem","before","actually","working","with","a","real","dataset",")",",","we","can","use","it","in","order","to","collect","a","dataset","to","later","be","used","to","train","the","actual","agent",".","The","collected","dataset",",","in","this","case",",","can","be","collected","either","by","randomly","acting","in","the","environment","(","only","running","in","heatup",")",",","or","alternatively","by","training","a","different","agent","in","the","environment","and","using","its","collected","data","as","a","dataset",".","If","an","experience","generating","agent","parameters","are","given",",","we","will","instantiate","this","agent","and","use","it","in","order","to","train","on","the","environment","and","then","use","this","dataset","to","actually","train","an","agent",".","Otherwise",",","we","will","collect","a","random","dataset",".",":","param","agent_params",":","the","parameters","of","the","agent","to","train","using","batch","RL",":","param","env_params",":","[","optional","]","environment","parameters",",","for","cases","where","we","want","to","first","collect","a","dataset",":","param","vis_params",":","visualization","parameters",":","param","preset_validation_params",":","preset","validation","parameters",",","to","be","used","for","testing","purposes",":","param","name",":","graph","name",":","param","spaces_definition",":","when","working","with","a","dataset",",","we","need","to","get","a","description","of","the","actual","state","and","action","spaces","of","the","problem",":","param","reward_model_num_epochs",":","the","number","of","epochs","to","go","over","the","dataset","for","training","a","reward","model","for","the","'direct","method","'","and","'doubly","robust","'","OPE","methods",".",":","param","train_to_eval_ratio",":","percentage","of","the","data","transitions","to","be","used","for","training","vs.","evaluation",".","i.e",".","a","value","of","0.8","means","~80","%","of","the","transitions","will","be","used","for","training","and","~20","%","will","be","used","for","evaluation","using","OPE",".",":","param","experience_generating_agent_params",":","[","optional","]","parameters","of","an","agent","to","be","trained","vs.","an","environment",",","whose","his","collected","experience","will","be","used","to","train","the","acutal","(","another",")","agent",":","param","experience_generating_schedule_params",":","[","optional","]","graph","scheduling","parameters","for","training","the","experience","generating","agent"]
docstr: Run off-policy evaluation estimators to evaluate the trained policy performance against the dataset
  :return:
docstr_tok: ["Run","off-policy","evaluation","estimators","to","evaluate","the","trained","policy","performance","against","the","dataset",":","return",":"]
code: def run_off_policy_evaluation(self):
      """
      Run off-policy evaluation estimators to evaluate the trained policy performance against the dataset
      :return:
      """
      self.level_managers[0].agents["agent"].run_off_policy_evaluation()
  
imports: from copy import deepcopy
  from typing import Tuple, List, Union
  from rl_coach.agents.dqn_agent import DQNAgentParameters
  from rl_coach.agents.nec_agent import NECAgentParameters
  from rl_coach.architectures.network_wrapper import NetworkWrapper
  from rl_coach.base_parameters import AgentParameters, VisualizationParameters, TaskParameters, PresetValidationParameters
  from rl_coach.core_types import RunPhase, TotalStepsCounter, TrainingSteps, EnvironmentEpisodes, EnvironmentSteps
  from rl_coach.environments.environment import EnvironmentParameters, Environment
  from rl_coach.graph_managers.graph_manager import ScheduleParameters
  from rl_coach.graph_managers.basic_rl_graph_manager import BasicRLGraphManager
  from rl_coach.level_manager import LevelManager
  from rl_coach.logger import screen
  from rl_coach.schedules import LinearSchedule
  from rl_coach.spaces import SpacesDefinition
  from rl_coach.utils import short_dynamic_import
  from rl_coach.memories.episodic import EpisodicExperienceReplayParameters
  from rl_coach.core_types import TimeTypes
label_desc: Use image node_id 5 for calling the BatchRLGraphManager obj's underlying member method code with example usage: obj.run_off_policy_evaluation() without return types