class_docstr: An AdaNet estimator builder.
class_docstr_tok: ["An","AdaNet","estimator","builder","."]
docstr: Returns hyperparameters, including any flag value overrides.
  
  In order to allow for automated hyperparameter tuning, model hyperparameters
  are aggregated within a tf.HParams object.  In this case, here are the
  hyperparameters and their descriptions:
  - optimizer: Name of the optimizer to use. See `optimizers.fn_with_name`.
  - learning_rate_schedule: Learning rate schedule string.
  - initial_learning_rate: The initial learning rate to use during training.
  - num_cells: Number of cells in the model. Must be divisible by 3.
  - num_conv_filters: The initial number of convolutional filters. The final
      layer will have 24*num_conv_filters channels.
  - weight_decay: Float amount of weight decay to apply to train loss.
  - use_aux_head: Whether to create an auxiliary head for training. This adds
      some non-determinism to training.
  - knowledge_distillation: Whether subnetworks should learn from the
      logits of the 'previous ensemble'/'previous subnetwork' in addition to
      the labels to distill/transfer/compress the knowledge in a manner
      inspired by Born Again Networks [Furlanello et al., 2018]
      (https://arxiv.org/abs/1805.04770) and Distilling the Knowledge in
      a Neural Network [Hinton at al., 2015]
      (https://arxiv.org/abs/1503.02531).
  - model_version: See `improve_nas.ModelVersion`.
  - adanet_lambda: See `adanet.Estimator`.
  - adanet_beta: See `adanet.Estimator`.
  - generator: Type of generator. `simple` generator is just ensembling,
      `dynamic` generator gradually grows the network.
  - boosting_iterations: The number of boosting iterations to perform. The
    final ensemble will have at most this many subnetworks comprising it.
  - evaluator_batch_size: Batch size for the evaluator to use when comparing
      candidates.
  - evaluator_steps: Number of batches for the evaluator to use when
      comparing candidates.
  - learn_mixture_weights: Whether to learn adanet mixture weights.
  - mixture_weight_type: Type of mxture weights.
  - batch_size: Batch size for training.
  - force_grow: Force AdaNet to add a candidate in each itteration, even if it
      would decreases the performance of the ensemble.
  - label_smoothing: Strength of label smoothing that will be applied (even
      non true labels will have a non zero representation in one hot encoding
      when computing loss).
  - clip_gradients: Clip gradient to this value.
  - aux_head_weight: NASNet cell parameter. Weight of auxiliary loss.
  - stem_multiplier: NASNet cell parameter.
  - drop_path_keep_prob: NASNet cell parameter. Propability for drop_path
      regularization.
  - dense_dropout_keep_prob: NASNet cell parameter. Dropout keep probability.
  - filter_scaling_rate: NASNet cell parameter. Controls growth of number of
      filters.
  - num_reduction_layers: NASNet cell parameter. Number of reduction layers
      that will be added to the architecture.
  - data_format: NASNet cell parameter. Controls whether data is in channels
      last or channels first format.
  - skip_reduction_layer_input: NASNet cell parameter. Whether to skip
      reduction layer.
  - use_bounded_activation: NASNet cell parameter. Whether to use bounded
      activations.
  - use_evaluator: Boolean whether to use the adanet.Evaluator to choose the
      best ensemble at each round.
  
  Args:
    default_batch_size: The default batch_size specified for training.
    hparams_string: If the hparams_string is given, then it will use any
      values specified in hparams to override any individually-set
      hyperparameter. This logic allows tuners to override hyperparameter
      settings to find optimal values.
  
  Returns:
    The hyperparameters as a tf.HParams object.
docstr_tok: ["Returns","hyperparameters",",","including","any","flag","value","overrides",".","In","order","to","allow","for","automated","hyperparameter","tuning",",","model","hyperparameters","are","aggregated","within","a","tf.HParams","object",".","In","this","case",",","here","are","the","hyperparameters","and","their","descriptions",":","-","optimizer",":","Name","of","the","optimizer","to","use",".","See","`","optimizers.fn_with_name","`",".","-","learning_rate_schedule",":","Learning","rate","schedule","string",".","-","initial_learning_rate",":","The","initial","learning","rate","to","use","during","training",".","-","num_cells",":","Number","of","cells","in","the","model",".","Must","be","divisible","by","3",".","-","num_conv_filters",":","The","initial","number","of","convolutional","filters",".","The","final","layer","will","have","24","*","num_conv_filters","channels",".","-","weight_decay",":","Float","amount","of","weight","decay","to","apply","to","train","loss",".","-","use_aux_head",":","Whether","to","create","an","auxiliary","head","for","training",".","This","adds","some","non-determinism","to","training",".","-","knowledge_distillation",":","Whether","subnetworks","should","learn","from","the","logits","of","the","'previous","ensemble'\/'previous","subnetwork","'","in","addition","to","the","labels","to","distill\/transfer\/compress","the","knowledge","in","a","manner","inspired","by","Born","Again","Networks","[","Furlanello","et","al.",",","2018","]","(","https",":","\/\/arxiv.org\/abs\/1805.04770",")","and","Distilling","the","Knowledge","in","a","Neural","Network","[","Hinton","at","al.",",","2015","]","(","https",":","\/\/arxiv.org\/abs\/1503.02531",")",".","-","model_version",":","See","`","improve_nas.ModelVersion","`",".","-","adanet_lambda",":","See","`","adanet.Estimator","`",".","-","adanet_beta",":","See","`","adanet.Estimator","`",".","-","generator",":","Type","of","generator",".","`","simple","`","generator","is","just","ensembling",",","`","dynamic","`","generator","gradually","grows","the","network",".","-","boosting_iterations",":","The","number","of","boosting","iterations","to","perform",".","The","final","ensemble","will","have","at","most","this","many","subnetworks","comprising","it",".","-","evaluator_batch_size",":","Batch","size","for","the","evaluator","to","use","when","comparing","candidates",".","-","evaluator_steps",":","Number","of","batches","for","the","evaluator","to","use","when","comparing","candidates",".","-","learn_mixture_weights",":","Whether","to","learn","adanet","mixture","weights",".","-","mixture_weight_type",":","Type","of","mxture","weights",".","-","batch_size",":","Batch","size","for","training",".","-","force_grow",":","Force","AdaNet","to","add","a","candidate","in","each","itteration",",","even","if","it","would","decreases","the","performance","of","the","ensemble",".","-","label_smoothing",":","Strength","of","label","smoothing","that","will","be","applied","(","even","non","true","labels","will","have","a","non","zero","representation","in","one","hot","encoding","when","computing","loss",")",".","-","clip_gradients",":","Clip","gradient","to","this","value",".","-","aux_head_weight",":","NASNet","cell","parameter",".","Weight","of","auxiliary","loss",".","-","stem_multiplier",":","NASNet","cell","parameter",".","-","drop_path_keep_prob",":","NASNet","cell","parameter",".","Propability","for","drop_path","regularization",".","-","dense_dropout_keep_prob",":","NASNet","cell","parameter",".","Dropout","keep","probability",".","-","filter_scaling_rate",":","NASNet","cell","parameter",".","Controls","growth","of","number","of","filters",".","-","num_reduction_layers",":","NASNet","cell","parameter",".","Number","of","reduction","layers","that","will","be","added","to","the","architecture",".","-","data_format",":","NASNet","cell","parameter",".","Controls","whether","data","is","in","channels","last","or","channels","first","format",".","-","skip_reduction_layer_input",":","NASNet","cell","parameter",".","Whether","to","skip","reduction","layer",".","-","use_bounded_activation",":","NASNet","cell","parameter",".","Whether","to","use","bounded","activations",".","-","use_evaluator",":","Boolean","whether","to","use","the","adanet.Evaluator","to","choose","the","best","ensemble","at","each","round",".","Args",":","default_batch_size",":","The","default","batch_size","specified","for","training",".","hparams_string",":","If","the","hparams_string","is","given",",","then","it","will","use","any","values","specified","in","hparams","to","override","any","individually-set","hyperparameter",".","This","logic","allows","tuners","to","override","hyperparameter","settings","to","find","optimal","values",".","Returns",":","The","hyperparameters","as","a","tf.HParams","object","."]
code: def hparams(self, default_batch_size, hparams_string):
      """Returns hyperparameters, including any flag value overrides.
  
      In order to allow for automated hyperparameter tuning, model hyperparameters
      are aggregated within a tf.HParams object.  In this case, here are the
      hyperparameters and their descriptions:
      - optimizer: Name of the optimizer to use. See `optimizers.fn_with_name`.
      - learning_rate_schedule: Learning rate schedule string.
      - initial_learning_rate: The initial learning rate to use during training.
      - num_cells: Number of cells in the model. Must be divisible by 3.
      - num_conv_filters: The initial number of convolutional filters. The final
          layer will have 24*num_conv_filters channels.
      - weight_decay: Float amount of weight decay to apply to train loss.
      - use_aux_head: Whether to create an auxiliary head for training. This adds
          some non-determinism to training.
      - knowledge_distillation: Whether subnetworks should learn from the
          logits of the 'previous ensemble'/'previous subnetwork' in addition to
          the labels to distill/transfer/compress the knowledge in a manner
          inspired by Born Again Networks [Furlanello et al., 2018]
          (https://arxiv.org/abs/1805.04770) and Distilling the Knowledge in
          a Neural Network [Hinton at al., 2015]
          (https://arxiv.org/abs/1503.02531).
      - model_version: See `improve_nas.ModelVersion`.
      - adanet_lambda: See `adanet.Estimator`.
      - adanet_beta: See `adanet.Estimator`.
      - generator: Type of generator. `simple` generator is just ensembling,
          `dynamic` generator gradually grows the network.
      - boosting_iterations: The number of boosting iterations to perform. The
        final ensemble will have at most this many subnetworks comprising it.
      - evaluator_batch_size: Batch size for the evaluator to use when comparing
          candidates.
      - evaluator_steps: Number of batches for the evaluator to use when
          comparing candidates.
      - learn_mixture_weights: Whether to learn adanet mixture weights.
      - mixture_weight_type: Type of mxture weights.
      - batch_size: Batch size for training.
      - force_grow: Force AdaNet to add a candidate in each itteration, even if it
          would decreases the performance of the ensemble.
      - label_smoothing: Strength of label smoothing that will be applied (even
          non true labels will have a non zero representation in one hot encoding
          when computing loss).
      - clip_gradients: Clip gradient to this value.
      - aux_head_weight: NASNet cell parameter. Weight of auxiliary loss.
      - stem_multiplier: NASNet cell parameter.
      - drop_path_keep_prob: NASNet cell parameter. Propability for drop_path
          regularization.
      - dense_dropout_keep_prob: NASNet cell parameter. Dropout keep probability.
      - filter_scaling_rate: NASNet cell parameter. Controls growth of number of
          filters.
      - num_reduction_layers: NASNet cell parameter. Number of reduction layers
          that will be added to the architecture.
      - data_format: NASNet cell parameter. Controls whether data is in channels
          last or channels first format.
      - skip_reduction_layer_input: NASNet cell parameter. Whether to skip
          reduction layer.
      - use_bounded_activation: NASNet cell parameter. Whether to use bounded
          activations.
      - use_evaluator: Boolean whether to use the adanet.Evaluator to choose the
          best ensemble at each round.
  
      Args:
        default_batch_size: The default batch_size specified for training.
        hparams_string: If the hparams_string is given, then it will use any
          values specified in hparams to override any individually-set
          hyperparameter. This logic allows tuners to override hyperparameter
          settings to find optimal values.
  
      Returns:
        The hyperparameters as a tf.HParams object.
      """
      hparams = tf.contrib.training.HParams(
          # Nasnet config hparams (default cifar config)
          num_cells=3,
          num_conv_filters=10,
          aux_head_weight=0.4,
          stem_multiplier=3.0,
          drop_path_keep_prob=0.6,
          use_aux_head=True,
          dense_dropout_keep_prob=1.0,
          filter_scaling_rate=2.0,
          num_reduction_layers=2,
          data_format="NHWC",
          skip_reduction_layer_input=0,
          use_bounded_activation=False,
          # Other hparams
          clip_gradients=5,
          optimizer="momentum",
          learning_rate_schedule="cosine",
          initial_learning_rate=0.025,
          weight_decay=5e-4,
          label_smoothing=0.1,
          knowledge_distillation=improve_nas.KnowledgeDistillation.ADAPTIVE,
          model_version="cifar",
          adanet_lambda=0.0,
          adanet_beta=0.0,
          generator=GeneratorType.SIMPLE,
          boosting_iterations=3,
          force_grow=True,
          evaluator_batch_size=-1,
          evaluator_steps=-1,
          batch_size=default_batch_size,
          learn_mixture_weights=False,
          mixture_weight_type=adanet.MixtureWeightType.SCALAR,
          use_evaluator=True,
      )
      if hparams_string:
          hparams = hparams.parse(hparams_string)
      if hparams.evaluator_batch_size < 0:
          hparams.evaluator_batch_size = default_batch_size
      if hparams.evaluator_steps < 0:
          hparams.evaluator_steps = None
      return hparams
  
file_path: adanet/research/improve_nas/trainer/adanet_improve_nas.py
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import adanet
  import tensorflow.compat.v1
label_desc: Use image node_id 2 for calling the Builder obj's underlying member method code with example usage: obj.hparams(default_batch_size, hparams_string) and returns: hparams