nas-search/search_main.py
def host_call_fn(gs, loss, lr, runtime, 
              t5x5_1, t50c_1, t100c_1, t5x5_2, t50c_2, t100c_2, 
              t5x5_3, t50c_3, t100c_3, t5x5_4, t50c_4, t100c_4, 
              t5x5_5, t50c_5, t100c_5, t5x5_6, t50c_6, t100c_6, 
              t5x5_7, t50c_7, t100c_7, t5x5_8, t50c_8, t100c_8, 
              t5x5_9, t50c_9, t100c_9, t5x5_10, t50c_10, t100c_10, 
              t5x5_11, t50c_11, t100c_11, t5x5_12, t50c_12, t100c_12, 
              t5x5_13, t50c_13, t100c_13, t5x5_14, t50c_14, t100c_14, 
              t5x5_15, t50c_15, t100c_15, t5x5_16, t50c_16, t100c_16, 
              t5x5_17, t50c_17, t100c_17, t5x5_18, t50c_18, t100c_18, 
              t5x5_19, t50c_19, t100c_19, t5x5_20, t50c_20, t100c_20):
        """Training host call. Creates scalar summaries for training metrics.

        This function is executed on the CPU and should not directly reference
        any Tensors in the rest of the `model_fn`. To pass Tensors from the
        model to the `metric_fn`, provide as part of the `host_call`. See
        https://www.tensorflow.org/api_docs/python/tf/contrib/tpu/TPUEstimatorSpec
        for more information.

        Arguments should match the list of `Tensor` objects passed as the second
        element in the tuple passed to `host_call`.

        Args:
          gs: `Tensor with shape `[batch]` for the global_step
          loss: `Tensor` with shape `[batch]` for the training loss.
          lr: `Tensor` with shape `[batch]` for the learning_rate.
          ce: `Tensor` with shape `[batch]` for the current_epoch.

        Returns:
          List of summary ops to run on the CPU host.
        """
        gs = gs[0]

        t_list = [[t5x5_1, t50c_1, t100c_1], [t5x5_2, t50c_2, t100c_2], 
              [t5x5_3, t50c_3, t100c_3], [t5x5_4, t50c_4, t100c_4], 
              [t5x5_5, t50c_5, t100c_5], [t5x5_6, t50c_6, t100c_6], 
              [t5x5_7, t50c_7, t100c_7], [t5x5_8, t50c_8, t100c_8], 
              [t5x5_9, t50c_9, t100c_9], [t5x5_10, t50c_10, t100c_10], 
              [t5x5_11, t50c_11, t100c_11], [t5x5_12, t50c_12, t100c_12], 
              [t5x5_13, t50c_13, t100c_13], [t5x5_14, t50c_14, t100c_14], 
              [t5x5_15, t50c_15, t100c_15], [t5x5_16, t50c_16, t100c_16], 
              [t5x5_17, t50c_17, t100c_17], [t5x5_18, t50c_18, t100c_18], 
              [t5x5_19, t50c_19, t100c_19], [t5x5_20, t50c_20, t100c_20]]

        # Host call fns are executed FLAGS.iterations_per_loop times after one
        # TPU loop is finished, setting max_queue value to the same as number of
        # iterations will make the summary writer only flush the data to storage
        # once per loop.
        with tf.contrib.summary.create_file_writer(
            FLAGS.model_dir, max_queue=FLAGS.iterations_per_loop).as_default():
          with tf.contrib.summary.always_record_summaries():
            tf.contrib.summary.scalar('loss', loss[0], step=gs)
            tf.contrib.summary.scalar('learning_rate', lr[0], step=gs)
            #tf.contrib.summary.scalar('current_epoch', ce[0], step=gs)
            tf.contrib.summary.scalar('runtime_ms', runtime[0], step=gs)
            for idx, t_ in enumerate(t_list):
              for label_, t_tensor in zip(['t5x5_','t50c_','t100c_'], t_):
                sum_label_ = label_ + str(idx+1) 
                tf.contrib.summary.scalar(sum_label_, t_tensor[0], step=gs)

            return tf.contrib.summary.all_summary_ops()
Training host call. Creates scalar summaries for training metrics.

        This function is executed on the CPU and should not directly reference
        any Tensors in the rest of the `model_fn`. To pass Tensors from the
        model to the `metric_fn`, provide as part of the `host_call`. See
        https://www.tensorflow.org/api_docs/python/tf/contrib/tpu/TPUEstimatorSpec
        for more information.

        Arguments should match the list of `Tensor` objects passed as the second
        element in the tuple passed to `host_call`.

        Args:
          gs: `Tensor with shape `[batch]` for the global_step
          loss: `Tensor` with shape `[batch]` for the training loss.
          lr: `Tensor` with shape `[batch]` for the learning_rate.
          ce: `Tensor` with shape `[batch]` for the current_epoch.

        Returns:
          List of summary ops to run on the CPU host.
        
Training host call. Creates scalar summaries for training metrics.
This function is executed on the CPU and should not directly reference
any Tensors in the rest of the `model_fn`. To pass Tensors from the
model to the `metric_fn`, provide as part of the `host_call`.

Arguments should match the list of `Tensor` objects passed as the second
element in the tuple passed to `host_call`.
Training host call . Creates scalar summaries for training metrics . This function is executed on the CPU and should not directly reference any Tensors in the rest of the ` model_fn ` . To pass Tensors from the model to the ` metric_fn ` provide as part of the ` host_call ` . Arguments should match the list of ` Tensor ` objects passed as the second element in the tuple passed to ` host_call ` .
def host_call_fn(gs, loss, lr, runtime, 
              t5x5_1, t50c_1, t100c_1, t5x5_2, t50c_2, t100c_2, 
              t5x5_3, t50c_3, t100c_3, t5x5_4, t50c_4, t100c_4, 
              t5x5_5, t50c_5, t100c_5, t5x5_6, t50c_6, t100c_6, 
              t5x5_7, t50c_7, t100c_7, t5x5_8, t50c_8, t100c_8, 
              t5x5_9, t50c_9, t100c_9, t5x5_10, t50c_10, t100c_10, 
              t5x5_11, t50c_11, t100c_11, t5x5_12, t50c_12, t100c_12, 
              t5x5_13, t50c_13, t100c_13, t5x5_14, t50c_14, t100c_14, 
              t5x5_15, t50c_15, t100c_15, t5x5_16, t50c_16, t100c_16, 
              t5x5_17, t50c_17, t100c_17, t5x5_18, t50c_18, t100c_18, 
              t5x5_19, t50c_19, t100c_19, t5x5_20, t50c_20, t100c_20):
        gs = gs[0]
        t_list = [[t5x5_1, t50c_1, t100c_1], [t5x5_2, t50c_2, t100c_2], 
              [t5x5_3, t50c_3, t100c_3], [t5x5_4, t50c_4, t100c_4], 
              [t5x5_5, t50c_5, t100c_5], [t5x5_6, t50c_6, t100c_6], 
              [t5x5_7, t50c_7, t100c_7], [t5x5_8, t50c_8, t100c_8], 
              [t5x5_9, t50c_9, t100c_9], [t5x5_10, t50c_10, t100c_10], 
              [t5x5_11, t50c_11, t100c_11], [t5x5_12, t50c_12, t100c_12], 
              [t5x5_13, t50c_13, t100c_13], [t5x5_14, t50c_14, t100c_14], 
              [t5x5_15, t50c_15, t100c_15], [t5x5_16, t50c_16, t100c_16], 
              [t5x5_17, t50c_17, t100c_17], [t5x5_18, t50c_18, t100c_18], 
              [t5x5_19, t50c_19, t100c_19], [t5x5_20, t50c_20, t100c_20]]
        with tf.contrib.summary.create_file_writer(
            FLAGS.model_dir, max_queue=FLAGS.iterations_per_loop).as_default():
          with tf.contrib.summary.always_record_summaries():
            tf.contrib.summary.scalar('loss', loss[0], step=gs)
            tf.contrib.summary.scalar('learning_rate', lr[0], step=gs)
            tf.contrib.summary.scalar('runtime_ms', runtime[0], step=gs)
            for idx, t_ in enumerate(t_list):
              for label_, t_tensor in zip(['t5x5_','t50c_','t100c_'], t_):
                sum_label_ = label_ + str(idx+1) 
                tf.contrib.summary.scalar(sum_label_, t_tensor[0], step=gs)
            return tf.contrib.summary.all_summary_ops()
def host_call_fn ( gs , loss , lr , runtime , t5x5_1 , t50c_1 , t100c_1 , t5x5_2 , t50c_2 , t100c_2 , t5x5_3 , t50c_3 , t100c_3 , t5x5_4 , t50c_4 , t100c_4 , t5x5_5 , t50c_5 , t100c_5 , t5x5_6 , t50c_6 , t100c_6 , t5x5_7 , t50c_7 , t100c_7 , t5x5_8 , t50c_8 , t100c_8 , t5x5_9 , t50c_9 , t100c_9 , t5x5_10 , t50c_10 , t100c_10 , t5x5_11 , t50c_11 , t100c_11 , t5x5_12 , t50c_12 , t100c_12 , t5x5_13 , t50c_13 , t100c_13 , t5x5_14 , t50c_14 , t100c_14 , t5x5_15 , t50c_15 , t100c_15 , t5x5_16 , t50c_16 , t100c_16 , t5x5_17 , t50c_17 , t100c_17 , t5x5_18 , t50c_18 , t100c_18 , t5x5_19 , t50c_19 , t100c_19 , t5x5_20 , t50c_20 , t100c_20 ) : gs = gs [ 0 ] t_list = [ [ t5x5_1 , t50c_1 , t100c_1 ] , [ t5x5_2 , t50c_2 , t100c_2 ] , [ t5x5_3 , t50c_3 , t100c_3 ] , [ t5x5_4 , t50c_4 , t100c_4 ] , [ t5x5_5 , t50c_5 , t100c_5 ] , [ t5x5_6 , t50c_6 , t100c_6 ] , [ t5x5_7 , t50c_7 , t100c_7 ] , [ t5x5_8 , t50c_8 , t100c_8 ] , [ t5x5_9 , t50c_9 , t100c_9 ] , [ t5x5_10 , t50c_10 , t100c_10 ] , [ t5x5_11 , t50c_11 , t100c_11 ] , [ t5x5_12 , t50c_12 , t100c_12 ] , [ t5x5_13 , t50c_13 , t100c_13 ] , [ t5x5_14 , t50c_14 , t100c_14 ] , [ t5x5_15 , t50c_15 , t100c_15 ] , [ t5x5_16 , t50c_16 , t100c_16 ] , [ t5x5_17 , t50c_17 , t100c_17 ] , [ t5x5_18 , t50c_18 , t100c_18 ] , [ t5x5_19 , t50c_19 , t100c_19 ] , [ t5x5_20 , t50c_20 , t100c_20 ] ] with tf . contrib . summary . create_file_writer ( FLAGS . model_dir , max_queue = FLAGS . iterations_per_loop ) . as_default ( ) : with tf . contrib . summary . always_record_summaries ( ) : tf . contrib . summary . scalar ( 'loss' , loss [ 0 ] , step = gs ) tf . contrib . summary . scalar ( 'learning_rate' , lr [ 0 ] , step = gs ) tf . contrib . summary . scalar ( 'runtime_ms' , runtime [ 0 ] , step = gs ) for idx , t_ in enumerate ( t_list ) : for label_ , t_tensor in zip ( [ 't5x5_' , 't50c_' , 't100c_' ] , t_ ) : sum_label_ = label_ + str ( idx + 1 ) tf . contrib . summary . scalar ( sum_label_ , t_tensor [ 0 ] , step = gs ) return tf . contrib . summary . all_summary_ops ( )
Training host call.
Training host call .
"""Training host call. Creates scalar summaries for training metrics.

        This function is executed on the CPU and should not directly reference
        any Tensors in the rest of the `model_fn`. To pass Tensors from the
        model to the `metric_fn`, provide as part of the `host_call`. See
        https://www.tensorflow.org/api_docs/python/tf/contrib/tpu/TPUEstimatorSpec
        for more information.

        Arguments should match the list of `Tensor` objects passed as the second
        element in the tuple passed to `host_call`.

        Args:
          gs: `Tensor with shape `[batch]` for the global_step
          loss: `Tensor` with shape `[batch]` for the training loss.
          lr: `Tensor` with shape `[batch]` for the learning_rate.
          ce: `Tensor` with shape `[batch]` for the current_epoch.

        Returns:
          List of summary ops to run on the CPU host.
        """ # Host call fns are executed FLAGS.iterations_per_loop times after one # TPU loop is finished, setting max_queue value to the same as number of # iterations will make the summary writer only flush the data to storage # once per loop. #tf.contrib.summary.scalar('current_epoch', ce[0], step=gs)
List of summary ops to run on the CPU host.
List of summary ops to run on the CPU host .
gs
`Tensor with shape `[batch]` for the global_step
` Tensor with shape ` [ batch ] ` for the global_step
loss
`Tensor` with shape `[batch]` for the training loss.
` Tensor ` with shape ` [ batch ] ` for the training loss .
lr
`Tensor` with shape `[batch]` for the learning_rate.
` Tensor ` with shape ` [ batch ] ` for the learning_rate .
runtime
t5x5_1
t50c_1
t100c_1
t5x5_2
t50c_2
t100c_2
t5x5_3
t50c_3
t100c_3
t5x5_4
t50c_4
t100c_4
t5x5_5
t50c_5
t100c_5
t5x5_6
t50c_6
t100c_6
t5x5_7
t50c_7
t100c_7
t5x5_8
t50c_8
t100c_8
t5x5_9
t50c_9
t100c_9
t5x5_10
t50c_10
t100c_10
t5x5_11
t50c_11
t100c_11
t5x5_12
t50c_12
t100c_12
t5x5_13
t50c_13
t100c_13
t5x5_14
t50c_14
t100c_14
t5x5_15
t50c_15
t100c_15
t5x5_16
t50c_16
t100c_16
t5x5_17
t50c_17
t100c_17
t5x5_18
t50c_18
t100c_18
t5x5_19
t50c_19
t100c_19
t5x5_20
t50c_20
t100c_20
ce
`Tensor` with shape `[batch]` for the current_epoch.
` Tensor ` with shape ` [ batch ] ` for the current_epoch .