class_docstr: PUBDEV-1843: Grid testing.  Subtask 2.
  
  This class is created to test the gridsearch for GBM algo and make sure it runs.  Only one test is performed
  here.
  
  Test Descriptions:
      a. grab all truely griddable parameters and randomly or manually set the parameter values.
      b. Next, build H2O GBM models using grid search.  Count and make sure models
         are only built for hyper-parameters set to legal values.  No model is built for bad hyper-parameters
         values.  We should instead get a warning/error message printed out.
      c. For each model built using grid search, we will extract the parameters used in building
         that model and manually build a H2O GBM model.  Training metrics are calculated from the
         gridsearch model and the manually built model.  If their metrics
         differ by too much, print a warning message but don't fail the test.
      d. we will check and make sure the models are built within the max_runtime_secs time limit that was set
         for it as well.  If max_runtime_secs was exceeded, declare test failure.
  
  Note that for hyper-parameters containing all legal parameter names and parameter value lists with legal
  and illegal values, grid-models should be built for all combinations of legal parameter values.  For
  illegal parameter values, a warning/error message should be printed out to warn the user but the
  program should not throw an exception;
  
  We will re-use the dataset generation methods for GLM.  There will be two type of datasets, one for regression and
  one for classification.
class_docstr_tok: ["PUBDEV-1843",":","Grid","testing",".","Subtask","2",".","This","class","is","created","to","test","the","gridsearch","for","GBM","algo","and","make","sure","it","runs",".","Only","one","test","is","performed","here",".","Test","Descriptions",":","a.","grab","all","truely","griddable","parameters","and","randomly","or","manually","set","the","parameter","values",".","b",".","Next",",","build","H2O","GBM","models","using","grid","search",".","Count","and","make","sure","models","are","only","built","for","hyper-parameters","set","to","legal","values",".","No","model","is","built","for","bad","hyper-parameters","values",".","We","should","instead","get","a","warning\/error","message","printed","out",".","c.","For","each","model","built","using","grid","search",",","we","will","extract","the","parameters","used","in","building","that","model","and","manually","build","a","H2O","GBM","model",".","Training","metrics","are","calculated","from","the","gridsearch","model","and","the","manually","built","model",".","If","their","metrics","differ","by","too","much",",","print","a","warning","message","but","do","n't","fail","the","test",".","d.","we","will","check","and","make","sure","the","models","are","built","within","the","max_runtime_secs","time","limit","that","was","set","for","it","as","well",".","If","max_runtime_secs","was","exceeded",",","declare","test","failure",".","Note","that","for","hyper-parameters","containing","all","legal","parameter","names","and","parameter","value","lists","with","legal","and","illegal","values",",","grid-models","should","be","built","for","all","combinations","of","legal","parameter","values",".","For","illegal","parameter","values",",","a","warning\/error","message","should","be","printed","out","to","warn","the","user","but","the","program","should","not","throw","an","exception",";","We","will","re-use","the","dataset","generation","methods","for","GLM",".","There","will","be","two","type","of","datasets",",","one","for","regression","and","one","for","classification","."]
docstr: This function setup the gridsearch hyper-parameters that will be used later on:
  
  1. It will first try to grab all the parameters that are griddable and parameters used by GBM.
  2. It will find the intersection of parameters that are both griddable and used by GBM.
  3. There are several extra parameters that are used by GBM that are denoted as griddable but actually is not.
  These parameters have to be discovered manually and they These are captured in self.exclude_parameter_lists.
  4. We generate the gridsearch hyper-parameter.  For numerical parameters, we will generate those randomly.
  For enums, we will include all of them.
  
  :return: None
docstr_tok: ["This","function","setup","the","gridsearch","hyper-parameters","that","will","be","used","later","on",":","1",".","It","will","first","try","to","grab","all","the","parameters","that","are","griddable","and","parameters","used","by","GBM",".","2",".","It","will","find","the","intersection","of","parameters","that","are","both","griddable","and","used","by","GBM",".","3",".","There","are","several","extra","parameters","that","are","used","by","GBM","that","are","denoted","as","griddable","but","actually","is","not",".","These","parameters","have","to","be","discovered","manually","and","they","These","are","captured","in","self.exclude_parameter_lists",".","4",".","We","generate","the","gridsearch","hyper-parameter",".","For","numerical","parameters",",","we","will","generate","those","randomly",".","For","enums",",","we","will","include","all","of","them",".",":","return",":","None"]
code: def setup_model(self):
      """
      This function setup the gridsearch hyper-parameters that will be used later on:
  
      1. It will first try to grab all the parameters that are griddable and parameters used by GBM.
      2. It will find the intersection of parameters that are both griddable and used by GBM.
      3. There are several extra parameters that are used by GBM that are denoted as griddable but actually is not.
      These parameters have to be discovered manually and they These are captured in self.exclude_parameter_lists.
      4. We generate the gridsearch hyper-parameter.  For numerical parameters, we will generate those randomly.
      For enums, we will include all of them.
  
      :return: None
      """
      # build bare bone model to get all parameters
      model = H2OGradientBoostingEstimator(
          distribution=self.family, seed=self.seed, nfolds=self.nfolds
      )
      model.train(
          x=self.x_indices,
          y=self.y_index,
          training_frame=self.training1_data,
      )
  
      self.model_run_time = pyunit_utils.find_grid_runtime(
          [model]
      )  # find model train time
      print(
          "Time taken to build a base barebone model is {0}".format(
              self.model_run_time
          )
      )
  
      summary_list = model._model_json["output"]["model_summary"]
      num_trees = summary_list["number_of_trees"][0]
  
      if num_trees == 0:
          self.min_runtime_per_tree = self.model_run_time
      else:
          self.min_runtime_per_tree = self.model_run_time / num_trees
  
      # grab all gridable parameters and its type
      (
          self.gridable_parameters,
          self.gridable_types,
          self.gridable_defaults,
      ) = pyunit_utils.get_gridables(model._model_json["parameters"])
  
      # randomly generate griddable parameters including values outside legal range, like setting alpha values to
      # be outside legal range of 0 and 1 and etc
      (
          self.hyper_params,
          self.gridable_parameters,
          self.gridable_types,
          self.gridable_defaults,
      ) = pyunit_utils.gen_grid_search(
          model.full_parameters.keys(),
          self.hyper_params,
          self.exclude_parameter_lists,
          self.gridable_parameters,
          self.gridable_types,
          self.gridable_defaults,
          random.randint(1, self.max_int_number),
          self.max_int_val,
          self.min_int_val,
          random.randint(1, self.max_real_number),
          self.max_real_val,
          self.min_real_val,
      )
  
      # scale the max_runtime_secs parameters
      time_scale = self.time_scale * self.model_run_time
      if "max_runtime_secs" in list(self.hyper_params):
          self.hyper_params["max_runtime_secs"] = [
              time_scale * x
              for x in self.hyper_params["max_runtime_secs"]
          ]
  
      # generate a new final_hyper_params which only takes a subset of all griddable parameters while
      # hyper_params take all griddable parameters and generate the grid search hyper-parameters
      [
          self.possible_number_models,
          self.final_hyper_params,
      ] = pyunit_utils.check_and_count_models(
          self.hyper_params,
          self.params_zero_one,
          self.params_more_than_zero,
          self.params_more_than_one,
          self.params_zero_positive,
          self.max_grid_model,
      )
  
      # must add max_runtime_secs to restrict unit test run time and as a promise to Arno to test for this
      if ("max_runtime_secs" not in list(self.final_hyper_params)) and (
          "max_runtime_secs" in list(self.hyper_params)
      ):
          self.final_hyper_params[
              "max_runtime_secs"
          ] = self.hyper_params["max_runtime_secs"]
          len_good_time = len(
              [
                  x
                  for x in self.hyper_params["max_runtime_secs"]
                  if (x >= 0)
              ]
          )
          self.possible_number_models = (
              self.possible_number_models * len_good_time
          )
  
      if "fold_assignment" in list(self.final_hyper_params):
          self.possible_number_models = (
              self.possible_number_models * self.scale_model
          )
  
      self.final_hyper_params["seed"] = [
          self.seed
      ]  # added see to make test more repeatable
  
      # write out the hyper-parameters used into json files.
      pyunit_utils.write_hyper_parameters_json(
          self.current_dir,
          self.sandbox_dir,
          self.json_filename,
          self.final_hyper_params,
      )
  
file_path: h2o-3/h2o-py/dynamic_tests/testdir_algos/gbm/pyunit_gbm_gridsearch_over_all_params_large.py
imports: import sys
  import random
  import os
  from builtins import range
  import time
  import json
  import h2o
  from tests import pyunit_utils
  from h2o.estimators.gbm import H2OGradientBoostingEstimator
  from h2o.grid.grid_search import H2OGridSearch
label_desc: Use image node_id 3 for calling the Test_gbm_grid_search obj's underlying member method code with example usage: obj.setup_model() without return types