class_docstr: Abstract class for custom training or validation loops.
  
  The scheduler will only be called at a batch or epoch frequency, depending
  on the user parameter. Be sure to set ``scheduler_step_freq`` in
  ``TorchTrainer`` to either "batch" or "epoch" to increment the scheduler
  correctly during training. If using a learning rate scheduler
  that depends on validation loss, you can use ``trainer.update_scheduler``.
  
  For both training and validation, there are two granularities that
  you can provide customization: per epoch or per batch.
  You do not need to override both.
  
  .. image:: raysgd-custom.jpg
      :scale: 80%
      :align: center
  
  Raises:
      ValueError if multiple models/optimizers/schedulers are provided.
          You are expected to subclass this class if you wish
          to train over multiple models/optimizers/schedulers.
code: def __init__(
      self,
      config,
      models,
      optimizers,
      world_rank,
      criterion=None,
      schedulers=None,
      use_fp16=False,
      use_tqdm=False,
  ):
      # You are not expected to override this method.
      self._models = models  # List of models
      assert isinstance(
          models, collections.Iterable
      ), "Components need to be iterable. Got: {}".format(type(models))
      self._optimizers = optimizers  # List of optimizers
      assert isinstance(
          optimizers, collections.Iterable
      ), "Components need to be iterable. Got: {}".format(
          type(optimizers)
      )
      self._world_rank = world_rank
      self._criterion = criterion
      self._schedulers = schedulers
      if schedulers:
          assert isinstance(
              schedulers, collections.Iterable
          ), "Components need to be iterable. Got: {}".format(
              type(schedulers)
          )
      self._config = config
      self._use_fp16 = use_fp16
      if tqdm is None and use_tqdm:
          raise ValueError(
              "tqdm must be installed to use tqdm in training."
          )
      self._use_tqdm = use_tqdm
      self.global_step = 0
  
      if type(self) is TrainingOperator:
          for component in (models, schedulers, optimizers):
              if _is_multiple(component):
                  raise ValueError(
                      "Need to provide a custom operator subclassing "
                      "TrainingOperator if using multi-scheduler, "
                      "multi-model or multi-optimizer training/validation."
                  )
      self.timers = TimerCollection()
      self.setup(config)
  
imports: import collections
  import torch
  import numpy
  from zoo.orca.learn.metrics import Metric
  from zoo.orca.learn.pytorch.utils import TimerCollection, AverageMeterCollection, NUM_SAMPLES
  from zoo.orca.learn.pytorch.constants import SCHEDULER_STEP_EPOCH, NUM_STEPS, SCHEDULER_STEP_BATCH, SCHEDULER_STEP
  from torch.nn.parallel import DistributedDataParallel
label_desc: Use image node_id 1 to create a new TrainingOperator object  with example: obj = TrainingOperator(config, models, optimizers, world_rank, criterion, schedulers, use_fp16, use_tqdm)