class_docstr: DeepSpeed trials are created by subclassing this abstract class.
  
  We can do the following things in this trial class:
  
  * **Define the DeepSpeed model engine which includes the model, optimizer, and lr_scheduler**.
  
     In the :meth:`__init__` method, initialize models and, optionally, optimizers and
     LR schedulers and pass them to ``deepspeed.initialize`` to build the model engine.  Then
     pass the created model engine to ``wrap_model_engine`` provided by
     :class:`~determined.pytorch.deepspeed.DeepSpeedTrialContext`.
     We support multiple DeepSpeed model engines if they only use data parallelism or if
     they use the same model parallel unit.
  
  * **Run forward and backward passes**.
  
     In :meth:`train_batch`, use the methods provided by the DeepSpeed model engine to perform
     the backward pass and optimizer step.  These methods will differ depending on whether
     you are using pipeline parallelism or not.
class_docstr_tok: ["DeepSpeed","trials","are","created","by","subclassing","this","abstract","class",".","We","can","do","the","following","things","in","this","trial","class",":","*","*","*","Define","the","DeepSpeed","model","engine","which","includes","the","model",",","optimizer",",","and","lr_scheduler","*","*",".","In","the",":","meth",":","`","__init__","`","method",",","initialize","models","and",",","optionally",",","optimizers","and","LR","schedulers","and","pass","them","to","``","deepspeed.initialize","``","to","build","the","model","engine",".","Then","pass","the","created","model","engine","to","``","wrap_model_engine","``","provided","by",":","class",":","`","~determined.pytorch.deepspeed.DeepSpeedTrialContext","`",".","We","support","multiple","DeepSpeed","model","engines","if","they","only","use","data","parallelism","or","if","they","use","the","same","model","parallel","unit",".","*","*","*","Run","forward","and","backward","passes","*","*",".","In",":","meth",":","`","train_batch","`",",","use","the","methods","provided","by","the","DeepSpeed","model","engine","to","perform","the","backward","pass","and","optimizer","step",".","These","methods","will","differ","depending","on","whether","you","are","using","pipeline","parallelism","or","not","."]
docstr: Defines a dictionary of string names to callbacks to be used during
  training and/or validation.
  
  The string name will be used as the key to save and restore callback
  state for any callback that defines :meth:`load_state_dict` and :meth:`state_dict`.
code: def build_callbacks(self) -> Dict[str, pytorch.PyTorchCallback]:
      """
      Defines a dictionary of string names to callbacks to be used during
      training and/or validation.
  
      The string name will be used as the key to save and restore callback
      state for any callback that defines :meth:`load_state_dict` and :meth:`state_dict`.
      """
      return {}
  
imports: import abc
  import contextlib
  import logging
  import os
  import pathlib
  import pickle
  import random
  import time
  import warnings
  from typing import Any, Callable, Dict, Iterator, List, Optional, Type, Union, cast
  import deepspeed
  import numpy
  import torch
  from deepspeed.runtime import dataloader
  import determined
  from determined import layers, pytorch, util, workload
  from determined.pytorch import deepspeed
  from determined.pytorch import dsat
label_desc: Use image node_id 5 for calling the DeepSpeedTrial obj's underlying member method code with example usage: obj.build_callbacks() and returns: dict