class_docstr: A class to wrap training code.
docstr: Fixes the shared parameters and updates the controller parameters.
  
  The controller is updated with a score function gradient estimator
  (i.e., REINFORCE), with the reward being c/valid_ppl, where valid_ppl
  is computed on a minibatch of validation data.
  
  A moving average baseline is used.
  
  The controller is trained for 2000 steps per epoch (i.e.,
  first (Train Shared) phase -> second (Train Controller) phase).
code: def train_controller(self):
      """Fixes the shared parameters and updates the controller parameters.
  
      The controller is updated with a score function gradient estimator
      (i.e., REINFORCE), with the reward being c/valid_ppl, where valid_ppl
      is computed on a minibatch of validation data.
  
      A moving average baseline is used.
  
      The controller is trained for 2000 steps per epoch (i.e.,
      first (Train Shared) phase -> second (Train Controller) phase).
      """
      model = self.controller
      model.train()
      # TODO(brendan): Why can't we call shared.eval() here? Leads to loss
      # being uniformly zero for the controller.
      # self.shared.eval()
  
      avg_reward_base = None
      baseline = None
      adv_history = []
      entropy_history = []
      reward_history = []
  
      hidden = self.shared.init_hidden(self.args.batch_size)
      total_loss = 0
      valid_idx = 0
      for step in range(self.args.controller_max_step):
          # sample models
          dags, log_probs, entropies = self.controller.sample(
              with_details=True
          )
  
          # calculate reward
          np_entropies = entropies.data.cpu().numpy()
          # NOTE(brendan): No gradients should be backpropagated to the
          # shared model during controller training, obviously.
          with _get_no_grad_ctx_mgr():
              rewards, hidden = self.get_reward(
                  dags, np_entropies, hidden, valid_idx
              )
  
          # discount
          if 1 > self.args.discount > 0:
              rewards = discount(rewards, self.args.discount)
  
          reward_history.extend(rewards)
          entropy_history.extend(np_entropies)
  
          # moving average baseline
          if baseline is None:
              baseline = rewards
          else:
              decay = self.args.ema_baseline_decay
              baseline = decay * baseline + (1 - decay) * rewards
  
          adv = rewards - baseline
          adv_history.extend(adv)
  
          # policy loss
          loss = -log_probs * utils.get_variable(
              adv, self.cuda, requires_grad=False
          )
          if self.args.entropy_mode == "regularizer":
              loss -= self.args.entropy_coeff * entropies
  
          loss = loss.sum()  # or loss.mean()
  
          # update
          self.controller_optim.zero_grad()
          loss.backward()
  
          if self.args.controller_grad_clip > 0:
              torch.nn.utils.clip_grad_norm(
                  model.parameters(), self.args.controller_grad_clip
              )
          self.controller_optim.step()
  
          total_loss += utils.to_item(loss.data)
  
          if ((step % self.args.log_step) == 0) and (step > 0):
              self._summarize_controller_train(
                  total_loss,
                  adv_history,
                  entropy_history,
                  reward_history,
                  avg_reward_base,
                  dags,
              )
  
              reward_history, adv_history, entropy_history = [], [], []
              total_loss = 0
  
          self.controller_step += 1
  
          prev_valid_idx = valid_idx
          valid_idx = (valid_idx + self.max_length) % (
              self.valid_data.size(0) - 1
          )
          # NOTE(brendan): Whenever we wrap around to the beginning of the
          # validation data, we reset the hidden states.
          if prev_valid_idx > valid_idx:
              hidden = self.shared.init_hidden(self.args.batch_size)
  
imports: import contextlib
  import glob
  import math
  import os
  import numpy
  import scipy.signal
  from tensorboard import TensorBoard
  import torch
  from torch import nn
  import torch.nn.parallel
  from torch.autograd import Variable
  import models
  import utils
label_desc: Use image node_id 7 for calling the Trainer obj's underlying member method code with example usage: obj.train_controller() without return types