class_docstr: User-facing API for accessing the NASBench dataset.
docstr: Fetch one of the evaluations for this model spec.
  
  Each call will sample one of the config['num_repeats'] evaluations of the
  model. This means that repeated queries of the same model (or isomorphic
  models) may return identical metrics.
  
  This function will increment the budget counters for benchmarking purposes.
  See self.training_time_spent, and self.total_epochs_spent.
  
  This function also allows querying the evaluation metrics at the halfway
  point of training using stop_halfway. Using this option will increment the
  budget counters only up to the halfway point.
  
  Args:
    model_spec: ModelSpec object.
    epochs: number of epochs trained. Must be one of the evaluated number of
      epochs, [4, 12, 36, 108] for the full dataset.
    stop_halfway: if True, returned dict will only contain the training time
      and accuracies at the halfway point of training (num_epochs/2).
      Otherwise, returns the time and accuracies at the end of training
      (num_epochs).
  
  Returns:
    dict containing the evaluated data for this object.
  
  Raises:
    OutOfDomainError: if model_spec or num_epochs is outside the search space.
docstr_tok: ["Fetch","one","of","the","evaluations","for","this","model","spec",".","Each","call","will","sample","one","of","the","config","[","'num_repeats","'","]","evaluations","of","the","model",".","This","means","that","repeated","queries","of","the","same","model","(","or","isomorphic","models",")","may","return","identical","metrics",".","This","function","will","increment","the","budget","counters","for","benchmarking","purposes",".","See","self.training_time_spent",",","and","self.total_epochs_spent",".","This","function","also","allows","querying","the","evaluation","metrics","at","the","halfway","point","of","training","using","stop_halfway",".","Using","this","option","will","increment","the","budget","counters","only","up","to","the","halfway","point",".","Args",":","model_spec",":","ModelSpec","object",".","epochs",":","number","of","epochs","trained",".","Must","be","one","of","the","evaluated","number","of","epochs",",","[","4",",","12",",","36",",","108","]","for","the","full","dataset",".","stop_halfway",":","if","True",",","returned","dict","will","only","contain","the","training","time","and","accuracies","at","the","halfway","point","of","training","(","num_epochs\/2",")",".","Otherwise",",","returns","the","time","and","accuracies","at","the","end","of","training","(","num_epochs",")",".","Returns",":","dict","containing","the","evaluated","data","for","this","object",".","Raises",":","OutOfDomainError",":","if","model_spec","or","num_epochs","is","outside","the","search","space","."]
code: def query(self, model_spec, epochs=108, stop_halfway=False):
      """Fetch one of the evaluations for this model spec.
  
      Each call will sample one of the config['num_repeats'] evaluations of the
      model. This means that repeated queries of the same model (or isomorphic
      models) may return identical metrics.
  
      This function will increment the budget counters for benchmarking purposes.
      See self.training_time_spent, and self.total_epochs_spent.
  
      This function also allows querying the evaluation metrics at the halfway
      point of training using stop_halfway. Using this option will increment the
      budget counters only up to the halfway point.
  
      Args:
        model_spec: ModelSpec object.
        epochs: number of epochs trained. Must be one of the evaluated number of
          epochs, [4, 12, 36, 108] for the full dataset.
        stop_halfway: if True, returned dict will only contain the training time
          and accuracies at the halfway point of training (num_epochs/2).
          Otherwise, returns the time and accuracies at the end of training
          (num_epochs).
  
      Returns:
        dict containing the evaluated data for this object.
  
      Raises:
        OutOfDomainError: if model_spec or num_epochs is outside the search space.
      """
      if epochs not in self.valid_epochs:
          raise OutOfDomainError(
              "invalid number of epochs, must be one of %s"
              % self.valid_epochs
          )
  
      fixed_stat, computed_stat = self.get_metrics_from_spec(model_spec)
      sampled_index = random.randint(0, self.config["num_repeats"] - 1)
      computed_stat = computed_stat[epochs][sampled_index]
  
      data = {}
      data["module_adjacency"] = fixed_stat["module_adjacency"]
      data["module_operations"] = fixed_stat["module_operations"]
      data["trainable_parameters"] = fixed_stat["trainable_parameters"]
  
      if stop_halfway:
          data["training_time"] = computed_stat["halfway_training_time"]
          data["train_accuracy"] = computed_stat[
              "halfway_train_accuracy"
          ]
          data["validation_accuracy"] = computed_stat[
              "halfway_validation_accuracy"
          ]
          data["test_accuracy"] = computed_stat["halfway_test_accuracy"]
      else:
          data["training_time"] = computed_stat["final_training_time"]
          data["train_accuracy"] = computed_stat["final_train_accuracy"]
          data["validation_accuracy"] = computed_stat[
              "final_validation_accuracy"
          ]
          data["test_accuracy"] = computed_stat["final_test_accuracy"]
  
      self.training_time_spent += data["training_time"]
      if stop_halfway:
          self.total_epochs_spent += epochs // 2
      else:
          self.total_epochs_spent += epochs
  
      return data
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import copy
  import random
  import time
  import shelve
  import hashlib
  import _pickle
  import numpy
label_desc: Use image node_id 2 for calling the NASBench obj's underlying member method code with example usage: obj.query(model_spec, epochs, stop_halfway) and returns: data