class_docstr: Automatic Machine Learning
  
  The Automatic Machine Learning (AutoML) function automates the supervised machine learning model training process.
  It trains several models, cross-validated by default, by using the following available algorithms:
  
  - XGBoost
  - GBM (Gradient Boosting Machine)
  - GLM (Generalized Linear Model)
  - DRF (Distributed Random Forest)
  - XRT (eXtremely Randomized Trees)
  - DeepLearning (Fully Connected Deep Neural Network)
  
  It also applies HPO on the following algorithms:
  
  - XGBoost
  - GBM
  - DeepLearning
  
  In some cases, there will not be enough time to complete all the algorithms, so some may be missing from the
  leaderboard. 
  Finally, AutoML also trains several Stacked Ensemble models at various stages during the run.
  Mainly two kinds of Stacked Ensemble models are trained:
  
  - one of all available models at time t
  - one of only the best models of each kind at time t.
  
  Note that Stacked Ensemble models are trained only if there isn't another stacked ensemble with the same base models.
  
  :examples:
  
  >>> import h2o
  >>> from h2o.automl import H2OAutoML
  >>> h2o.init()
  >>> # Import a sample binary outcome train/test set into H2O
  >>> train = h2o.import_file("https://s3.amazonaws.com/erin-data/higgs/higgs_train_10k.csv")
  >>> test = h2o.import_file("https://s3.amazonaws.com/erin-data/higgs/higgs_test_5k.csv")
  >>> # Identify the response and set of predictors
  >>> y = "response"
  >>> x = list(train.columns)  #if x is defined as all columns except the response, then x is not required
  >>> x.remove(y)
  >>> # For binary classification, response should be a factor
  >>> train[y] = train[y].asfactor()
  >>> test[y] = test[y].asfactor()
  >>> # Run AutoML for 30 seconds
  >>> aml = H2OAutoML(max_runtime_secs = 30)
  >>> aml.train(x = x, y = y, training_frame = train)
  >>> # Print Leaderboard (ranked by xval metrics)
  >>> aml.leaderboard
  >>> # (Optional) Evaluate performance on a test set
  >>> perf = aml.leader.model_performance(test)
  >>> perf.auc()
docstr: Create a new H2OAutoML instance.
  
  :param int nfolds: Specify a value >= 2 for the number of folds for k-fold cross-validation for the models in the AutoML or specify ``-1`` (default)
      to let AutoML choose what it will do. If the data is big enough (depending on the cluster resources), it will create a blending frame
      and will not do cross-validation. Otherwise, it will use 5 fold cross-validation.        
  :param bool balance_classes: Specify whether to oversample the minority classes to balance the class distribution. This option can increase
      the data frame size. This option is only applicable for classification. If the oversampled size of the dataset exceeds the maximum size
      calculated using the ``max_after_balance_size`` parameter, then the majority classes will be undersampled to satisfy the size limit.
      Defaults to ``False``.
  :param class_sampling_factors: Desired over/under-sampling ratios per class (in lexicographic order).
      If not specified, sampling factors will be automatically computed to obtain class balance during training. Requires ``balance_classes`` set to ``True``.
  :param float max_after_balance_size: Maximum relative size of the training data after balancing class counts (can be less than 1.0).
      Requires ``balance_classes``.
      Defaults to ``5.0``.
  :param int max_runtime_secs: Specify the maximum time that the AutoML process will run for.
      If both ``max_runtime_secs`` and ``max_models`` are specified, then the AutoML run will stop as soon as it hits either of these limits.
      If neither ``max_runtime_secs`` nor ``max_models`` are specified, then ``max_runtime_secs`` dynamically
      defaults to 3600 seconds (1 hour). Otherwise, defaults to ``0`` (no limit).
  :param int max_runtime_secs_per_model: Controls the max time the AutoML run will dedicate to each individual model.
      Defaults to ``0`` (disabled: no time limit).
      Note that models constrained by a time budget are not guaranteed reproducible.
  :param int max_models: Specify the maximum number of models to build in an AutoML run, excluding the Stacked Ensemble models.
      Defaults to ``None`` (disabled: no limitation).
      Always set this parameter to ensure AutoML reproducibility: all models are then trained until convergence and none is constrained by a time budget.
  :param Union[str, dict] distribution: Distribution function used by algorithms that support it; other algorithms
      use their defaults.  Possible values: "AUTO", "bernoulli", "multinomial", "gaussian", "poisson", "gamma",
      "tweedie", "laplace", "quantile", "huber", "custom", and for parameterized distributions dictionary form is
      used to specify the parameter, e.g., ``dict(type="tweedie", tweedie_power=1.5)``.
      Defaults to ``AUTO``.
  :param str stopping_metric: Specifies the metric to use for early stopping. 
      The available options are:
      
          - ``"AUTO"`` (This defaults to ``"logloss"`` for classification, ``"deviance"`` for regression)
          - ``"deviance"``
          - ``"logloss"``
          - ``"mse"``
          - ``"rmse"``
          - ``"mae"``
          - ``"rmsle"``
          - ``"auc"``
          - ``aucpr``
          - ``"lift_top_group"``
          - ``"misclassification"``
          - ``"mean_per_class_error"``
          - ``"r2"``
          
      Defaults to ``"AUTO"``.
  :param float stopping_tolerance: Specify the relative tolerance for the metric-based stopping criterion to stop a grid search and
      the training of individual models within the AutoML run.
      Defaults to ``0.001`` if the dataset is at least 1 million rows;
      otherwise it defaults to a value determined by the size of the dataset and the non-NA-rate, in which case the value is computed as 1/sqrt(nrows * non-NA-rate).
  :param int stopping_rounds: Stop training new models in the AutoML run when the option selected for
      ``stopping_metric`` doesn't improve for the specified number of models, based on a simple moving average.
      To disable this feature, set it to ``0``.
      Defaults to ``3`` and must be an non-negative integer.
  :param int seed: Set a seed for reproducibility. 
      AutoML can only guarantee reproducibility if ``max_models`` or early stopping is used because ``max_runtime_secs`` is resource limited, 
      meaning that if the resources are not the same between runs, AutoML may be able to train more models on one run vs another.
      In addition, H2O Deep Learning models are not reproducible by default for performance reasons, so ``exclude_algos`` must contain ``DeepLearning``.
      Defaults to ``None``.
  :param str project_name: Character string to identify an AutoML project.
      Defaults to ``None``, which means a project name will be auto-generated based on the training frame ID.
      More models can be trained on an existing AutoML project by specifying the same project name in multiple calls to the AutoML function
      (as long as the same training frame, or a sample, is used in subsequent runs).
  :param exclude_algos: List the algorithms to skip during the model-building phase. 
      The full list of options is:
      
          - ``"DRF"`` (Random Forest and Extremely-Randomized Trees)
          - ``"GLM"``
          - ``"XGBoost"``
          - ``"GBM"``
          - ``"DeepLearning"``
          - ``"StackedEnsemble"``
          
      Defaults to ``None``, which means that all appropriate H2O algorithms will be used, if the search stopping criteria allow. Optional.
      Usage example::
      
          exclude_algos = ["GLM", "DeepLearning", "DRF"]
          
  :param include_algos: List the algorithms to restrict to during the model-building phase.
      This can't be used in combination with ``exclude_algos`` param.
      Defaults to ``None``, which means that all appropriate H2O algorithms will be used, if the search stopping criteria allow. Optional.
      Usage example::
  
          include_algos = ["GLM", "DeepLearning", "DRF"]
          
  :param exploitation_ratio: The budget ratio (between 0 and 1) dedicated to the exploitation (vs exploration) phase.
      By default, the exploitation phase is ``0`` (disabled) as this is still experimental;
      to activate it, it is recommended to try a ratio around 0.1.
      Note that the current exploitation phase only tries to fine-tune the best XGBoost and the best GBM found during exploration.
  :param modeling_plan: List of modeling steps to be used by the AutoML engine (they may not all get executed, depending on other constraints).
      Defaults to ``None`` (Expert usage only).
  :param preprocessing: List of preprocessing steps to run. Only ``["target_encoding"]`` is currently supported. Experimental.
  :param monotone_constraints: A mapping that represents monotonic constraints.
      Use ``+1`` to enforce an increasing constraint and ``-1`` to specify a decreasing constraint.
  :param keep_cross_validation_predictions: Whether to keep the predictions of the cross-validation predictions.
      This needs to be set to ``True`` if running the same AutoML object for repeated runs because CV predictions are required to build 
      additional Stacked Ensemble models in AutoML. 
      Defaults to ``False``.
  :param keep_cross_validation_models: Whether to keep the cross-validated models.
      Keeping cross-validation models may consume significantly more memory in the H2O cluster.
      Defaults to ``False``.
  :param keep_cross_validation_fold_assignment: Whether to keep fold assignments in the models.
      Deleting them will save memory in the H2O cluster. 
      Defaults to ``False``.
  :param sort_metric: Metric to sort the leaderboard by at the end of an AutoML run. 
      For binomial classification, select from the following options:
      
          - ``"auc"``
          - ``"aucpr"``
          - ``"logloss"``
          - ``"mean_per_class_error"``
          - ``"rmse"``
          - ``"mse"``
          
      For multinomial classification, select from the following options:
      
          - ``"mean_per_class_error"``
          - ``"logloss"``
          - ``"rmse"``
          - ``"mse"``
          
      For regression, select from the following options:
  
          - ``"deviance"``
          - ``"rmse"``
          - ``"mse"``
          - ``"mae"``
          - ``"rmlse"``
          
      Defaults to ``"AUTO"`` (This translates to ``"auc"`` for binomial classification, ``"mean_per_class_error"`` for multinomial classification, ``"deviance"`` for regression).
  :param custom_metric_func: Reference to custom evaluation function, format: `language:keyName=funcName`
         Defaults to ``None``.
  :type custom_metric_func: str, optional
  :param export_checkpoints_dir: Path to a directory where every model will be stored in binary form.
  :param verbosity: Verbosity of the backend messages printed during training.
      Available options are ``None`` (live log disabled), ``"debug"``, ``"info"``, ``"warn"`` or ``"error"``.
      Defaults to ``"warn"``.
code: def __init__(
      self,
      nfolds=-1,
      balance_classes=False,
      class_sampling_factors=None,
      max_after_balance_size=5.0,
      max_runtime_secs=None,
      max_runtime_secs_per_model=None,
      max_models=None,
      distribution="AUTO",
      stopping_metric="AUTO",
      stopping_tolerance=None,
      stopping_rounds=3,
      seed=None,
      project_name=None,
      exclude_algos=None,
      include_algos=None,
      exploitation_ratio=-1,
      modeling_plan=None,
      preprocessing=None,
      monotone_constraints=None,
      keep_cross_validation_predictions=False,
      keep_cross_validation_models=False,
      keep_cross_validation_fold_assignment=False,
      sort_metric="AUTO",
      custom_metric_func=None,
      export_checkpoints_dir=None,
      verbosity="warn",
      **kwargs,
  ):
      """
      Create a new H2OAutoML instance.
  
      :param int nfolds: Specify a value >= 2 for the number of folds for k-fold cross-validation for the models in the AutoML or specify ``-1`` (default)
          to let AutoML choose what it will do. If the data is big enough (depending on the cluster resources), it will create a blending frame
          and will not do cross-validation. Otherwise, it will use 5 fold cross-validation.
      :param bool balance_classes: Specify whether to oversample the minority classes to balance the class distribution. This option can increase
          the data frame size. This option is only applicable for classification. If the oversampled size of the dataset exceeds the maximum size
          calculated using the ``max_after_balance_size`` parameter, then the majority classes will be undersampled to satisfy the size limit.
          Defaults to ``False``.
      :param class_sampling_factors: Desired over/under-sampling ratios per class (in lexicographic order).
          If not specified, sampling factors will be automatically computed to obtain class balance during training. Requires ``balance_classes`` set to ``True``.
      :param float max_after_balance_size: Maximum relative size of the training data after balancing class counts (can be less than 1.0).
          Requires ``balance_classes``.
          Defaults to ``5.0``.
      :param int max_runtime_secs: Specify the maximum time that the AutoML process will run for.
          If both ``max_runtime_secs`` and ``max_models`` are specified, then the AutoML run will stop as soon as it hits either of these limits.
          If neither ``max_runtime_secs`` nor ``max_models`` are specified, then ``max_runtime_secs`` dynamically
          defaults to 3600 seconds (1 hour). Otherwise, defaults to ``0`` (no limit).
      :param int max_runtime_secs_per_model: Controls the max time the AutoML run will dedicate to each individual model.
          Defaults to ``0`` (disabled: no time limit).
          Note that models constrained by a time budget are not guaranteed reproducible.
      :param int max_models: Specify the maximum number of models to build in an AutoML run, excluding the Stacked Ensemble models.
          Defaults to ``None`` (disabled: no limitation).
          Always set this parameter to ensure AutoML reproducibility: all models are then trained until convergence and none is constrained by a time budget.
      :param Union[str, dict] distribution: Distribution function used by algorithms that support it; other algorithms
          use their defaults.  Possible values: "AUTO", "bernoulli", "multinomial", "gaussian", "poisson", "gamma",
          "tweedie", "laplace", "quantile", "huber", "custom", and for parameterized distributions dictionary form is
          used to specify the parameter, e.g., ``dict(type="tweedie", tweedie_power=1.5)``.
          Defaults to ``AUTO``.
      :param str stopping_metric: Specifies the metric to use for early stopping.
          The available options are:
  
              - ``"AUTO"`` (This defaults to ``"logloss"`` for classification, ``"deviance"`` for regression)
              - ``"deviance"``
              - ``"logloss"``
              - ``"mse"``
              - ``"rmse"``
              - ``"mae"``
              - ``"rmsle"``
              - ``"auc"``
              - ``aucpr``
              - ``"lift_top_group"``
              - ``"misclassification"``
              - ``"mean_per_class_error"``
              - ``"r2"``
  
          Defaults to ``"AUTO"``.
      :param float stopping_tolerance: Specify the relative tolerance for the metric-based stopping criterion to stop a grid search and
          the training of individual models within the AutoML run.
          Defaults to ``0.001`` if the dataset is at least 1 million rows;
          otherwise it defaults to a value determined by the size of the dataset and the non-NA-rate, in which case the value is computed as 1/sqrt(nrows * non-NA-rate).
      :param int stopping_rounds: Stop training new models in the AutoML run when the option selected for
          ``stopping_metric`` doesn't improve for the specified number of models, based on a simple moving average.
          To disable this feature, set it to ``0``.
          Defaults to ``3`` and must be an non-negative integer.
      :param int seed: Set a seed for reproducibility.
          AutoML can only guarantee reproducibility if ``max_models`` or early stopping is used because ``max_runtime_secs`` is resource limited,
          meaning that if the resources are not the same between runs, AutoML may be able to train more models on one run vs another.
          In addition, H2O Deep Learning models are not reproducible by default for performance reasons, so ``exclude_algos`` must contain ``DeepLearning``.
          Defaults to ``None``.
      :param str project_name: Character string to identify an AutoML project.
          Defaults to ``None``, which means a project name will be auto-generated based on the training frame ID.
          More models can be trained on an existing AutoML project by specifying the same project name in multiple calls to the AutoML function
          (as long as the same training frame, or a sample, is used in subsequent runs).
      :param exclude_algos: List the algorithms to skip during the model-building phase.
          The full list of options is:
  
              - ``"DRF"`` (Random Forest and Extremely-Randomized Trees)
              - ``"GLM"``
              - ``"XGBoost"``
              - ``"GBM"``
              - ``"DeepLearning"``
              - ``"StackedEnsemble"``
  
          Defaults to ``None``, which means that all appropriate H2O algorithms will be used, if the search stopping criteria allow. Optional.
          Usage example::
  
              exclude_algos = ["GLM", "DeepLearning", "DRF"]
  
      :param include_algos: List the algorithms to restrict to during the model-building phase.
          This can't be used in combination with ``exclude_algos`` param.
          Defaults to ``None``, which means that all appropriate H2O algorithms will be used, if the search stopping criteria allow. Optional.
          Usage example::
  
              include_algos = ["GLM", "DeepLearning", "DRF"]
  
      :param exploitation_ratio: The budget ratio (between 0 and 1) dedicated to the exploitation (vs exploration) phase.
          By default, the exploitation phase is ``0`` (disabled) as this is still experimental;
          to activate it, it is recommended to try a ratio around 0.1.
          Note that the current exploitation phase only tries to fine-tune the best XGBoost and the best GBM found during exploration.
      :param modeling_plan: List of modeling steps to be used by the AutoML engine (they may not all get executed, depending on other constraints).
          Defaults to ``None`` (Expert usage only).
      :param preprocessing: List of preprocessing steps to run. Only ``["target_encoding"]`` is currently supported. Experimental.
      :param monotone_constraints: A mapping that represents monotonic constraints.
          Use ``+1`` to enforce an increasing constraint and ``-1`` to specify a decreasing constraint.
      :param keep_cross_validation_predictions: Whether to keep the predictions of the cross-validation predictions.
          This needs to be set to ``True`` if running the same AutoML object for repeated runs because CV predictions are required to build
          additional Stacked Ensemble models in AutoML.
          Defaults to ``False``.
      :param keep_cross_validation_models: Whether to keep the cross-validated models.
          Keeping cross-validation models may consume significantly more memory in the H2O cluster.
          Defaults to ``False``.
      :param keep_cross_validation_fold_assignment: Whether to keep fold assignments in the models.
          Deleting them will save memory in the H2O cluster.
          Defaults to ``False``.
      :param sort_metric: Metric to sort the leaderboard by at the end of an AutoML run.
          For binomial classification, select from the following options:
  
              - ``"auc"``
              - ``"aucpr"``
              - ``"logloss"``
              - ``"mean_per_class_error"``
              - ``"rmse"``
              - ``"mse"``
  
          For multinomial classification, select from the following options:
  
              - ``"mean_per_class_error"``
              - ``"logloss"``
              - ``"rmse"``
              - ``"mse"``
  
          For regression, select from the following options:
  
              - ``"deviance"``
              - ``"rmse"``
              - ``"mse"``
              - ``"mae"``
              - ``"rmlse"``
  
          Defaults to ``"AUTO"`` (This translates to ``"auc"`` for binomial classification, ``"mean_per_class_error"`` for multinomial classification, ``"deviance"`` for regression).
      :param custom_metric_func: Reference to custom evaluation function, format: `language:keyName=funcName`
             Defaults to ``None``.
      :type custom_metric_func: str, optional
      :param export_checkpoints_dir: Path to a directory where every model will be stored in binary form.
      :param verbosity: Verbosity of the backend messages printed during training.
          Available options are ``None`` (live log disabled), ``"debug"``, ``"info"``, ``"warn"`` or ``"error"``.
          Defaults to ``"warn"``.
      """
  
      # early validate kwargs, extracting hidden parameters:
      algo_parameters = {}
      for k in kwargs:
          if k == "algo_parameters":
              algo_parameters = kwargs[k] or {}
          else:
              raise TypeError(
                  "H2OAutoML got an unexpected keyword argument '%s'"
                  % k
              )
  
      # Check if H2O jar contains AutoML
      try:
          h2o.api("GET /3/Metadata/schemas/AutoMLV99")
      except h2o.exceptions.H2OResponseError as e:
          print(e)
          print(
              "*******************************************************************\n"
              "*Please verify that your H2O jar has the proper AutoML extensions.*\n"
              "*******************************************************************\n"
              "\nVerbose Error Message:"
          )
  
      self._job = None
      self._leader_id = None
      self._leaderboard = None
      self._verbosity = verbosity
      self._event_log = None
      self._training_info = None
      self._state_json = None
      self._build_resp = (
          None  # contains all the actual parameters used on backend
      )
  
      self.__frozen = False
      self.__input = (
          dict()
      )  # contains all the input params as entered by the user
  
      # Make bare minimum params containers
      self.build_control = dict()
      self.build_models = dict()
      self.input_spec = dict()
  
      self.project_name = project_name
      self.nfolds = nfolds
      self.distribution = distribution
      self.custom_metric_func = custom_metric_func
      self.balance_classes = balance_classes
      self.class_sampling_factors = class_sampling_factors
      self.max_after_balance_size = max_after_balance_size
      self.keep_cross_validation_models = keep_cross_validation_models
      self.keep_cross_validation_fold_assignment = (
          keep_cross_validation_fold_assignment
      )
      self.keep_cross_validation_predictions = (
          keep_cross_validation_predictions
      )
      self.export_checkpoints_dir = export_checkpoints_dir
  
      self.max_runtime_secs = max_runtime_secs
      self.max_runtime_secs_per_model = max_runtime_secs_per_model
      self.max_models = max_models
      self.stopping_metric = stopping_metric
      self.stopping_tolerance = stopping_tolerance
      self.stopping_rounds = stopping_rounds
      self.seed = seed
      self.exclude_algos = exclude_algos
      self.include_algos = include_algos
      self.exploitation_ratio = exploitation_ratio
      self.modeling_plan = modeling_plan
      self.preprocessing = preprocessing
      if monotone_constraints is not None:
          algo_parameters["monotone_constraints"] = monotone_constraints
      self._algo_parameters = algo_parameters
  
      self.sort_metric = sort_metric
  
imports: import functools
  from inspect import getdoc
  import re
  import h2o
  from h2o.base import Keyed
  from h2o.estimators import H2OEstimator
  from h2o.exceptions import H2OResponseError, H2OValueError
  from h2o.frame import H2OFrame
  from h2o.job import H2OJob
  from h2o.utils.compatibility import *
  from h2o.utils.shared_utils import check_id
  from h2o.utils.typechecks import assert_is_type, is_type, numeric
  from ._base import H2OAutoMLBaseMixin, _fetch_state
label_desc: Use image node_id 1 to create a new H2OAutoML object from inherited base classes: H2OAutoMLBaseMixin, Keyed with example: obj = H2OAutoML(nfolds, balance_classes, class_sampling_factors, max_after_balance_size, max_runtime_secs, max_runtime_secs_per_model, max_models, distribution, stopping_metric, stopping_tolerance, stopping_rounds, seed, project_name, exclude_algos, include_algos, exploitation_ratio, modeling_plan, preprocessing, monotone_constraints, keep_cross_validation_predictions, keep_cross_validation_models, keep_cross_validation_fold_assignment, sort_metric, custom_metric_func, export_checkpoints_dir, verbosity)