class_docstr: PUBDEV-1843: Grid testing.  Subtask 7,8.
  
  This class is created to test the gridsearch with the GLM algo using Guassian, Binonmial or
  Multinomial family.  Three tests are written to test the following conditions:
  1. For hyper-parameters containing all legal parameter names and parameter value lists with legal
  and illegal values, grid-models should be built for all combinations of legal parameter values.  For
  illegal parameter values, a warning/error message should be printed out to warn the user but the
  program should not throw an exception;
  2. For hyper-parameters with illegal names, an exception should be thrown and no models should be built;
  3. For parameters that are specified both in the hyper-parameters and model parameters, unless the values
  specified in the model parameters are set to default values, an exception will be thrown since parameters are
  not supposed to be specified in both places.
  
  Test Descriptions:
  test1_glm_grid_search_over_params: test for condition 1 and performs the following:
      a. grab all truely griddable parameters and randomly or manually set the parameter values.
      b. Next, build H2O GLM models using grid search.  Count and make sure models
         are only built for hyper-parameters set to legal values.  No model is built for bad hyper-parameters
         values.  We should instead get a warning/error message printed out.
      c. For each model built using grid search, we will extract the parameters used in building
         that model and manually build a H2O GLM model.  Training metrics are calculated from the
         gridsearch model and the manually built model.  If their metrics
         differ by too much, print a warning message but don't fail the test.
      d. we will check and make sure the models are built within the max_runtime_secs time limit that was set
         for it as well.  If max_runtime_secs was exceeded, declare test failure.
  
  test2_illegal_name_value: test for condition 1 and 2.  Randomly go into the hyper_parameters that we
  have specified, either
      a. randomly alter the name of a hyper-parameter name (fatal, exception will be thrown)
      b. randomly choose a hyper-parameter and remove all elements in its list (fatal)
      c. add randomly generated new hyper-parameter names with random list (fatal)
      d: randomly choose a hyper-parameter and insert an illegal type into it (non fatal, model built with
         legal hyper-parameters settings only and error messages printed out for illegal hyper-parameters
         settings)
  
  test3_duplicated_parameter_specification: test for condition 3.  Go into our hyper_parameters list, randomly
  choose some hyper-parameters to specify and specify it as part of the model parameters.  Hence, the same
  parameter is specified both in the model parameters and hyper-parameters.  Make sure the test failed with
  error messages when the parameter values are not set to default if they are specified in the model parameters
  as well as in the hyper-parameters.
class_docstr_tok: ["PUBDEV-1843",":","Grid","testing",".","Subtask","7,8",".","This","class","is","created","to","test","the","gridsearch","with","the","GLM","algo","using","Guassian",",","Binonmial","or","Multinomial","family",".","Three","tests","are","written","to","test","the","following","conditions",":","1",".","For","hyper-parameters","containing","all","legal","parameter","names","and","parameter","value","lists","with","legal","and","illegal","values",",","grid-models","should","be","built","for","all","combinations","of","legal","parameter","values",".","For","illegal","parameter","values",",","a","warning\/error","message","should","be","printed","out","to","warn","the","user","but","the","program","should","not","throw","an","exception",";","2",".","For","hyper-parameters","with","illegal","names",",","an","exception","should","be","thrown","and","no","models","should","be","built",";","3",".","For","parameters","that","are","specified","both","in","the","hyper-parameters","and","model","parameters",",","unless","the","values","specified","in","the","model","parameters","are","set","to","default","values",",","an","exception","will","be","thrown","since","parameters","are","not","supposed","to","be","specified","in","both","places",".","Test","Descriptions",":","test1_glm_grid_search_over_params",":","test","for","condition","1","and","performs","the","following",":","a.","grab","all","truely","griddable","parameters","and","randomly","or","manually","set","the","parameter","values",".","b",".","Next",",","build","H2O","GLM","models","using","grid","search",".","Count","and","make","sure","models","are","only","built","for","hyper-parameters","set","to","legal","values",".","No","model","is","built","for","bad","hyper-parameters","values",".","We","should","instead","get","a","warning\/error","message","printed","out",".","c.","For","each","model","built","using","grid","search",",","we","will","extract","the","parameters","used","in","building","that","model","and","manually","build","a","H2O","GLM","model",".","Training","metrics","are","calculated","from","the","gridsearch","model","and","the","manually","built","model",".","If","their","metrics","differ","by","too","much",",","print","a","warning","message","but","do","n't","fail","the","test",".","d.","we","will","check","and","make","sure","the","models","are","built","within","the","max_runtime_secs","time","limit","that","was","set","for","it","as","well",".","If","max_runtime_secs","was","exceeded",",","declare","test","failure",".","test2_illegal_name_value",":","test","for","condition","1","and","2",".","Randomly","go","into","the","hyper_parameters","that","we","have","specified",",","either","a.","randomly","alter","the","name","of","a","hyper-parameter","name","(","fatal",",","exception","will","be","thrown",")","b.","randomly","choose","a","hyper-parameter","and","remove","all","elements","in","its","list","(","fatal",")","c.","add","randomly","generated","new","hyper-parameter","names","with","random","list","(","fatal",")","d",":","randomly","choose","a","hyper-parameter","and","insert","an","illegal","type","into","it","(","non","fatal",",","model","built","with","legal","hyper-parameters","settings","only","and","error","messages","printed","out","for","illegal","hyper-parameters","settings",")","test3_duplicated_parameter_specification",":","test","for","condition","3",".","Go","into","our","hyper_parameters","list",",","randomly","choose","some","hyper-parameters","to","specify","and","specify","it","as","part","of","the","model","parameters",".","Hence",",","the","same","parameter","is","specified","both","in","the","model","parameters","and","hyper-parameters",".","Make","sure","the","test","failed","with","error","messages","when","the","parameter","values","are","not","set","to","default","if","they","are","specified","in","the","model","parameters","as","well","as","in","the","hyper-parameters","."]
code: def tear_down(self):
      pyunit_utils.remove_files(
          os.path.join(self.current_dir, self.json_filename)
      )
      pyunit_utils.remove_files(
          os.path.join(self.current_dir, self.json_filename_bad)
      )
  
file_path: h2o-3/h2o-py/dynamic_tests/testdir_algos/glm/pyunit_glm_gridsearch_over_all_params_large.py
imports: import sys
  import random
  import os
  import numpy
  from builtins import range
  import time
  import h2o
  from tests import pyunit_utils
  from h2o.estimators.glm import H2OGeneralizedLinearEstimator
  from h2o.grid.grid_search import H2OGridSearch
label_desc: Use image node_id 4 for calling the Test_glm_grid_search obj's underlying member method code with example usage: obj.tear_down() without return types