hunk.code_window:             else:
                   del active_episodes[env_id]
                   resetted_obs: Dict[AgentID, EnvObsType] = base_env.try_reset(
                       env_id)
               # Reset not supported, drop this env from the ready list.
               if resetted_obs is None:
                   if horizon != float("inf"):
                       raise ValueError(
  
hunk.labels: keep keep replace keep keep keep keep keep
hunk.after_edit:                 resetted_obs: Dict[EnvID, Dict[AgentID, EnvObsType]] = \
                       base_env.try_reset(
  
hunk.file_path: rllib/evaluation/sampler.py
hunk.type: replace
file: .. _trainer-reference-docs:
  
  Trainer API
  ===========
  
  The :py:class:`~ray.rllib.agents.trainer.Trainer` class is the highest-level API in RLlib.
  It allows you to train and evaluate policies, save an experiment's progress and restore from
  a prior saved experiment when continuing an RL run.
  :py:class:`~ray.rllib.agents.trainer.Trainer` is a sub-class
  of :py:class:`~ray.tune.Trainable`
  and thus fully supports distributed hyperparameter tuning for RL.
  
  .. https://docs.google.com/drawings/d/1J0nfBMZ8cBff34e-nSPJZMM1jKOuUL11zFJm6CmWtJU/edit
  .. figure:: ../../images/rllib/trainer_class_overview.svg
      :align: left
  
      **A typical RLlib Trainer object:** The components sitting inside a Trainer are
      normally N :py:class:`~ray.rllib.evaluation.worker_set.WorkerSet`s
      (each consisting of one local :py:class:`~ray.rllib.evaluation.RolloutWorker`
      and zero or more \@ray.remote
      :py:class:`~ray.rllib.evaluation.RolloutWorker`s),
      a set of :py:class:`~ray.rllib.policy.Policy`(ies)
      and their NN models per worker, and a (already vectorized)
      RLlib :py:class:`~ray.rllib.env.base_env.BaseEnv` per worker.
  
  
  Building Custom Trainer Classes
  -------------------------------
  
  .. warning::
      As of Ray >= 1.9, it is no longer recommended to use the `build_trainer()` utility
      function for creating custom Trainer sub-classes.
      Instead, follow the simple guidelines here for directly sub-classing from
      :py:class:`~ray.rllib.agents.trainer.Trainer`.
  
  In order to create a custom Trainer, sub-class the
  :py:class:`~ray.rllib.agents.trainer.Trainer` class
  and override one or more of its methods. Those are in particular:
  
  * :py:meth:`~ray.rllib.agents.trainer.Trainer.get_default_config`
  * :py:meth:`~ray.rllib.agents.trainer.Trainer.validate_config`
  * :py:meth:`~ray.rllib.agents.trainer.Trainer.get_default_policy_class`
  * :py:meth:`~ray.rllib.agents.trainer.Trainer.setup`
  * :py:meth:`~ray.rllib.agents.trainer.Trainer.step_attempt`
  * :py:meth:`~ray.rllib.agents.trainer.Trainer.execution_plan`
  
  `See here for an example on how to override Trainer <https://github.com/ray-project/ray/blob/master/rllib/agents/pg/pg.py>`_.
  
  
  Trainer base class (ray.rllib.agents.trainer.Trainer)
  -----------------------------------------------------
  
  .. autoclass:: ray.rllib.agents.trainer.Trainer
      :special-members: __init__
      :members:
  
  ..  static-members: get_default_config, execution_plan
  