class_docstr: This class is created to test the GLM algo with Binomial family.  In this case, the relationship
  between the response Y and predictor vector X is assumed to be
  Prob(Y = 1|X) = exp(W^T * X + E)/(1+exp(W^T * X + E)) where E is unknown Gaussian noise.  We
  generate random data set using the exact formula.  To evaluate the H2O GLM Model, we run the sklearn
  logistic regression with the same data sets and compare the performance of the two.  If they are close
  enough within a certain tolerance, we declare the H2O model working.  When regularization and other
  parameters are enabled, we can evaluate H2O GLM model performance by comparing the logloss/accuracy
  from H2O model and to the H2O model generated without regularization.  As long as they do not deviate
  too much, we consider the H2O model performance satisfactory.
  
  In particular, I have written 8 tests in the hope to exercise as many parameters settings of the GLM
  algo with Binomial distribution as possible.  Tomas has requested 2 tests to be added to test his new
  feature of missing_values_handling with predictors with both categorical/real columns.  Here is a list
  of all tests descriptions:
  
  test1_glm_no_regularization(): sklearn logistic regression model is built.
      H2O GLM is built for Binomial family with the same random data sets.  We observe
      the weights, confusion matrices from the two models.  We compare the logloss, prediction
      accuracy from the two models to determine if H2O GLM model shall pass the test.
  test2_glm_lambda_search(): test lambda search with alpha set to 0.5 per Tomas's
      suggestion.  Make sure logloss and prediction accuracy generated here is comparable in
      value to H2O GLM with no regularization.
  test3_glm_grid_search_over_params(): test grid search over
      various alpha values while lambda is set to be the best value obtained
      from test 2.  Cross validation with k=5 and random assignment is enabled
      as well.  The best model performance hopefully will generate logloss and
      prediction accuracies close to H2O with no regularization in test 1.
  test4_glm_remove_collinear_columns(): test parameter remove_collinear_columns=True
      with lambda set to best lambda from test 2, alpha set to best alpha from Gridsearch
      and solver set to the one which generate the smallest validation logloss.  The same dataset
      is used here except that we randomly choose predictor columns to repeat and scale.
      Make sure logloss and prediction accuracies generated here is comparable in value
      to H2O GLM model with no regularization.
  test5_missing_values(): Test parameter missing_values_handling="MeanImputation" with
      only real value predictors.  The same data sets as before is used.  However, we
      go into the predictor matrix and randomly decide to replace a value with
      nan and create missing values.  Sklearn logistic regression model is built using the
      data set where we have imputed the missing values.  This Sklearn model will be used to
      compare our H2O models with.
  test6_enum_missing_values(): Test parameter missing_values_handling="MeanImputation" with
      mixed predictors (categorical/real value columns).  We first generate a data set that
      contains a random number of columns of categorical and real value columns.  Next, we
      encode the categorical columns.  Then, we generate the random data set using the formula
      as before.  Next, we go into the predictor matrix and randomly
      decide to change a value to be nan and create missing values.  Again, we build a Sklearn
      logistic regression model and compare our H2O model with it.
  test7_missing_enum_values_lambda_search(): Test parameter
      missing_values_handling="MeanImputation" with mixed predictors (categorical/real value columns).
      Test parameter missing_values_handling="MeanImputation" with
      mixed predictors (categorical/real value columns) and setting lambda search to be True.
      We use the same prediction data  with missing values from test6.  Next, we encode the categorical columns using
      true one hot encoding since Lambda-search will be enabled with alpha set to 0.5.  Since the encoding
      is different in this case from test6, we will build a brand new Sklearn logistic regression model and
      compare the best H2O model logloss/prediction accuracy with it.
class_docstr_tok: ["This","class","is","created","to","test","the","GLM","algo","with","Binomial","family",".","In","this","case",",","the","relationship","between","the","response","Y","and","predictor","vector","X","is","assumed","to","be","Prob","(","Y","=","1|X",")","=","exp","(","W^T","*","X","+","E",")","\/","(","1+exp","(","W^T","*","X","+","E",")",")","where","E","is","unknown","Gaussian","noise",".","We","generate","random","data","set","using","the","exact","formula",".","To","evaluate","the","H2O","GLM","Model",",","we","run","the","sklearn","logistic","regression","with","the","same","data","sets","and","compare","the","performance","of","the","two",".","If","they","are","close","enough","within","a","certain","tolerance",",","we","declare","the","H2O","model","working",".","When","regularization","and","other","parameters","are","enabled",",","we","can","evaluate","H2O","GLM","model","performance","by","comparing","the","logloss\/accuracy","from","H2O","model","and","to","the","H2O","model","generated","without","regularization",".","As","long","as","they","do","not","deviate","too","much",",","we","consider","the","H2O","model","performance","satisfactory",".","In","particular",",","I","have","written","8","tests","in","the","hope","to","exercise","as","many","parameters","settings","of","the","GLM","algo","with","Binomial","distribution","as","possible",".","Tomas","has","requested","2","tests","to","be","added","to","test","his","new","feature","of","missing_values_handling","with","predictors","with","both","categorical\/real","columns",".","Here","is","a","list","of","all","tests","descriptions",":","test1_glm_no_regularization","(",")",":","sklearn","logistic","regression","model","is","built",".","H2O","GLM","is","built","for","Binomial","family","with","the","same","random","data","sets",".","We","observe","the","weights",",","confusion","matrices","from","the","two","models",".","We","compare","the","logloss",",","prediction","accuracy","from","the","two","models","to","determine","if","H2O","GLM","model","shall","pass","the","test",".","test2_glm_lambda_search","(",")",":","test","lambda","search","with","alpha","set","to","0.5","per","Tomas's","suggestion",".","Make","sure","logloss","and","prediction","accuracy","generated","here","is","comparable","in","value","to","H2O","GLM","with","no","regularization",".","test3_glm_grid_search_over_params","(",")",":","test","grid","search","over","various","alpha","values","while","lambda","is","set","to","be","the","best","value","obtained","from","test","2",".","Cross","validation","with","k=5","and","random","assignment","is","enabled","as","well",".","The","best","model","performance","hopefully","will","generate","logloss","and","prediction","accuracies","close","to","H2O","with","no","regularization","in","test","1.","test4_glm_remove_collinear_columns","(",")",":","test","parameter","remove_collinear_columns=True","with","lambda","set","to","best","lambda","from","test","2",",","alpha","set","to","best","alpha","from","Gridsearch","and","solver","set","to","the","one","which","generate","the","smallest","validation","logloss",".","The","same","dataset","is","used","here","except","that","we","randomly","choose","predictor","columns","to","repeat","and","scale",".","Make","sure","logloss","and","prediction","accuracies","generated","here","is","comparable","in","value","to","H2O","GLM","model","with","no","regularization",".","test5_missing_values","(",")",":","Test","parameter","missing_values_handling=","''","MeanImputation","''","with","only","real","value","predictors",".","The","same","data","sets","as","before","is","used",".","However",",","we","go","into","the","predictor","matrix","and","randomly","decide","to","replace","a","value","with","nan","and","create","missing","values",".","Sklearn","logistic","regression","model","is","built","using","the","data","set","where","we","have","imputed","the","missing","values",".","This","Sklearn","model","will","be","used","to","compare","our","H2O","models","with",".","test6_enum_missing_values","(",")",":","Test","parameter","missing_values_handling=","''","MeanImputation","''","with","mixed","predictors","(","categorical\/real","value","columns",")",".","We","first","generate","a","data","set","that","contains","a","random","number","of","columns","of","categorical","and","real","value","columns",".","Next",",","we","encode","the","categorical","columns",".","Then",",","we","generate","the","random","data","set","using","the","formula","as","before",".","Next",",","we","go","into","the","predictor","matrix","and","randomly","decide","to","change","a","value","to","be","nan","and","create","missing","values",".","Again",",","we","build","a","Sklearn","logistic","regression","model","and","compare","our","H2O","model","with","it",".","test7_missing_enum_values_lambda_search","(",")",":","Test","parameter","missing_values_handling=","''","MeanImputation","''","with","mixed","predictors","(","categorical\/real","value","columns",")",".","Test","parameter","missing_values_handling=","''","MeanImputation","''","with","mixed","predictors","(","categorical\/real","value","columns",")","and","setting","lambda","search","to","be","True",".","We","use","the","same","prediction","data","with","missing","values","from","test6",".","Next",",","we","encode","the","categorical","columns","using","true","one","hot","encoding","since","Lambda-search","will","be","enabled","with","alpha","set","to","0.5",".","Since","the","encoding","is","different","in","this","case","from","test6",",","we","will","build","a","brand","new","Sklearn","logistic","regression","model","and","compare","the","best","H2O","model","logloss\/prediction","accuracy","with","it","."]
docstr: Test parameter missing_values_handling="MeanImputation" with
  mixed predictors (categorical/real value columns).  We first generate a data set that
  contains a random number of columns of categorical and real value columns.  Next, we
  encode the categorical columns.  Then, we generate the random data set using the formula
  as before.  Next, we go into the predictor matrix and randomly
  decide to change a value to be nan and create missing values.  Again, we build a Sklearn
  logistic regression and compare our H2O models with it.
docstr_tok: ["Test","parameter","missing_values_handling=","''","MeanImputation","''","with","mixed","predictors","(","categorical\/real","value","columns",")",".","We","first","generate","a","data","set","that","contains","a","random","number","of","columns","of","categorical","and","real","value","columns",".","Next",",","we","encode","the","categorical","columns",".","Then",",","we","generate","the","random","data","set","using","the","formula","as","before",".","Next",",","we","go","into","the","predictor","matrix","and","randomly","decide","to","change","a","value","to","be","nan","and","create","missing","values",".","Again",",","we","build","a","Sklearn","logistic","regression","and","compare","our","H2O","models","with","it","."]
code: def test6_enum_missing_values(self):
      """
      Test parameter missing_values_handling="MeanImputation" with
      mixed predictors (categorical/real value columns).  We first generate a data set that
      contains a random number of columns of categorical and real value columns.  Next, we
      encode the categorical columns.  Then, we generate the random data set using the formula
      as before.  Next, we go into the predictor matrix and randomly
      decide to change a value to be nan and create missing values.  Again, we build a Sklearn
      logistic regression and compare our H2O models with it.
      """
  
      # no regularization in this case, use reference level plus one-hot-encoding
      print(
          "*******************************************************************************************"
      )
      print("Test6: test the GLM with enum/real values.")
      h2o.cluster_info()
  
      # training result from python sklearn
      (
          p_weights,
          p_logloss_train,
          p_cm_train,
          p_accuracy_training,
          p_logloss_test,
          p_cm_test,
          p_accuracy_test,
      ) = self.sklearn_binomial_result(
          self.training_data_file_enum_nans,
          self.test_data_file_enum_nans,
          True,
          False,
      )
  
      # import training set and test set with missing values
      training_data = h2o.import_file(
          pyunit_utils.locate(self.training_data_file_enum_nans)
      )
      test_data = h2o.import_file(
          pyunit_utils.locate(self.test_data_file_enum_nans)
      )
  
      # change the categorical data using .asfactor()
      for ind in range(self.enum_col):
          training_data[ind] = training_data[ind].round().asfactor()
          test_data[ind] = test_data[ind].round().asfactor()
  
      num_col = training_data.ncol
      y_index = num_col - 1
      x_indices = list(range(y_index))
  
      # change response variables to be categorical
      training_data[y_index] = training_data[y_index].round().asfactor()
  
      # check to make sure all response classes are represented, otherwise, quit
      if training_data[y_index].nlevels()[0] < self.class_number:
          print(
              "Response classes are not represented in training dataset."
          )
          sys.exit(0)
  
      test_data[y_index] = test_data[y_index].round().asfactor()
  
      # generate H2O model
      model_h2o = H2OGeneralizedLinearEstimator(
          family=self.family,
          Lambda=0,
          missing_values_handling="MeanImputation",
      )
      model_h2o.train(
          x=x_indices, y=y_index, training_frame=training_data
      )
  
      h2o_model_test_metrics = model_h2o.model_performance(
          test_data=test_data
      )
  
      num_test_failed = self.test_failed
  
      # print out comparison results our H2O GLM with Sklearn model
      self.test_failed = pyunit_utils.extract_comparison_attributes_and_print_multinomial(
          model_h2o,
          h2o_model_test_metrics,
          self.family,
          "\nTest6 Done!",
          compare_att_str=[
              "\nComparing intercept and " "weights ....",
              "\nComparing logloss from training" " dataset ....",
              "\nComparing logloss from test" " dataset ....",
              "\nComparing confusion matrices from"
              " training dataset ....",
              "\nComparing confusion matrices from" " test dataset ...",
              "\nComparing accuracy from training" " dataset ....",
              "\nComparing accuracy from test" " dataset ....",
          ],
          h2o_att_str=[
              "H2O with enum/real values, "
              "no regularization and missing values"
              " intercept and weights: \n",
              "H2O with enum/real values, no "
              "regularization and missing values"
              " logloss from training dataset: ",
              "H2O with enum/real values, no"
              " regularization and missing values"
              " logloss from test dataset",
              "H2O with enum/real values, no"
              " regularization and missing values"
              " confusion matrix from training"
              " dataset: \n",
              "H2O with enum/real values, no"
              " regularization and missing values"
              " confusion matrix from test"
              " dataset: \n",
              "H2O with enum/real values, no"
              " regularization and missing values "
              "accuracy from training dataset: ",
              "H2O with enum/real values, no "
              "regularization and missing values"
              " accuracy from test dataset: ",
          ],
          template_att_str=[
              "Sklearn missing values intercept " "and weights: \n",
              "Sklearn with enum/real values, no"
              " regularization and missing values"
              " logloss from training dataset: ",
              "Sklearn with enum/real values, no "
              "regularization and missing values"
              " logloss from test dataset: ",
              "Sklearn with enum/real values, no "
              "regularization and missing values "
              "confusion matrix from training"
              " dataset: \n",
              "Sklearn with enum/real values, no "
              "regularization and missing values "
              "confusion matrix from test "
              "dataset: \n",
              "Sklearn with enum/real values, no "
              "regularization and missing values "
              "accuracy from training dataset: ",
              "Sklearn with enum/real values, no "
              "regularization and missing values "
              "accuracy from test dataset: ",
          ],
          att_str_fail=[
              "Intercept and weights are not equal!",
              "Logloss from training dataset differ" " too much!",
              "Logloss from test dataset differ too" " much!",
              "",
              "",
              "Accuracies from training dataset" " differ too much!",
              "Accuracies from test dataset differ" " too much!",
          ],
          att_str_success=[
              "Intercept and weights are close" " enough!",
              "Logloss from training dataset are" " close enough!",
              "Logloss from test dataset are close" " enough!",
              "",
              "",
              "Accuracies from training dataset are" " close enough!",
              "Accuracies from test dataset are" " close enough!",
          ],
          can_be_better_than_template=[
              True,
              True,
              True,
              True,
              True,
              True,
              True,
          ],
          just_print=[True, True, True, True, True, True, False],
          failed_test_number=self.test_failed,
          template_params=[
              p_weights,
              p_logloss_train,
              p_cm_train,
              p_accuracy_training,
              p_logloss_test,
              p_cm_test,
              p_accuracy_test,
          ],
          ignored_eps=self.ignored_eps,
          allowed_diff=self.allowed_diff,
      )
  
      h2o.cluster_info()
      # print out test results and update test_failed_array status to reflect if this test has failed
      self.test_failed_array[
          self.test_num
      ] += pyunit_utils.show_test_results(
          "test6_enum_missing_values", num_test_failed, self.test_failed
      )
      self.test_num += 1
  
imports: import sys
  import random
  import os
  import math
  import numpy
  import h2o
  import time
  from builtins import range
  from tests import pyunit_utils
  from h2o.estimators.glm import H2OGeneralizedLinearEstimator
  from h2o.grid.grid_search import H2OGridSearch
  from scipy import stats
  from sklearn.linear_model import LogisticRegression
  from sklearn import metrics
label_desc: Use image node_id 9 for calling the TestGLMBinomial obj's underlying member method code with example usage: obj.test6_enum_missing_values() without return types