class_docstr: A class to wrap training code.
docstr: Cycles through alternately training the shared parameters and the
  controller, as described in Section 2.2, Training ENAS and Deriving
  Architectures, of the paper.
  
  From the paper (for Penn Treebank):
  
  - In the first phase, shared parameters omega are trained for 400
    steps, each on a minibatch of 64 examples.
  
  - In the second phase, the controller's parameters are trained for 2000
    steps.
    
  Args:
      single (bool): If True it won't train the controller and use the
                     same dag instead of derive().
code: def train(self, single=False):
      """Cycles through alternately training the shared parameters and the
      controller, as described in Section 2.2, Training ENAS and Deriving
      Architectures, of the paper.
  
      From the paper (for Penn Treebank):
  
      - In the first phase, shared parameters omega are trained for 400
        steps, each on a minibatch of 64 examples.
  
      - In the second phase, the controller's parameters are trained for 2000
        steps.
  
      Args:
          single (bool): If True it won't train the controller and use the
                         same dag instead of derive().
      """
      dag = utils.load_dag(self.args) if single else None
  
      if self.args.shared_initial_step > 0:
          self.train_shared(self.args.shared_initial_step)
          self.train_controller()
  
      for self.epoch in range(self.start_epoch, self.args.max_epoch):
          # 1. Training the shared parameters omega of the child models
          self.train_shared(dag=dag)
  
          # 2. Training the controller parameters theta
          if not single:
              self.train_controller()
  
          if self.epoch % self.args.save_epoch == 0:
              with _get_no_grad_ctx_mgr():
                  best_dag = dag if dag else self.derive()
                  self.evaluate(
                      self.eval_data,
                      best_dag,
                      "val_best",
                      max_num=self.args.batch_size * 100,
                  )
              self.save_model()
  
          if self.epoch >= self.args.shared_decay_after:
              utils.update_lr(self.shared_optim, self.shared_lr)
  
imports: import contextlib
  import glob
  import math
  import os
  import numpy
  import scipy.signal
  from tensorboard import TensorBoard
  import torch
  from torch import nn
  import torch.nn.parallel
  from torch.autograd import Variable
  import models
  import utils
label_desc: Use image node_id 3 for calling the Trainer obj's underlying member method code with example usage: obj.train(single) without return types