docstr: This function trains and evaluates performance of all estimators available in the
  model library using cross validation. The output of this function is a score grid
  with average cross validated scores. Metrics evaluated during CV can be accessed
  using the ``get_metrics`` function. Custom metrics can be added or removed using
  ``add_metric`` and ``remove_metric`` function.
  
  Example
  -------
  >>> from pycaret.datasets import get_data
  >>> juice = get_data('juice')
  >>> from pycaret.classification import *
  >>> exp_name = setup(data = juice,  target = 'Purchase')
  >>> best_model = compare_models()
  
  
  include: list of str or scikit-learn compatible object, default = None
      To train and evaluate select models, list containing model ID or scikit-learn
      compatible object can be passed in include param. To see a list of all models
      available in the model library use the ``models`` function.
  
  
  exclude: list of str, default = None
      To omit certain models from training and evaluation, pass a list containing
      model id in the exclude parameter. To see a list of all models available
      in the model library use the ``models`` function.
  
  
  fold: int or scikit-learn compatible CV generator, default = None
      Controls cross-validation. If None, the CV generator in the ``fold_strategy``
      parameter of the ``setup`` function is used. When an integer is passed,
      it is interpreted as the 'n_splits' parameter of the CV generator in the
      ``setup`` function.
  
  
  round: int, default = 4
      Number of decimal places the metrics in the score grid will be rounded to.
  
  
  cross_validation: bool, default = True
      When set to False, metrics are evaluated on holdout set. ``fold`` param
      is ignored when cross_validation is set to False.
  
  
  sort: str, default = 'Accuracy'
      The sort order of the score grid. It also accepts custom metrics that are
      added through the ``add_metric`` function.
  
  
  n_select: int, default = 1
      Number of top_n models to return. For example, to select top 3 models use
      n_select = 3.
  
  
  budget_time: int or float, default = None
      If not None, will terminate execution of the function after budget_time
      minutes have passed and return results up to that point.
  
  
  turbo: bool, default = True
      When set to True, it excludes estimators with longer training times. To
      see which algorithms are excluded use the ``models`` function.
  
  
  errors: str, default = 'ignore'
      When set to 'ignore', will skip the model with exceptions and continue.
      If 'raise', will break the function when exceptions are raised.
  
  
  fit_kwargs: dict, default = {} (empty dict)
      Dictionary of arguments passed to the fit method of the model.
  
  
  groups: str or array-like, with shape (n_samples,), default = None
      Optional group labels when 'GroupKFold' is used for the cross validation.
      It takes an array with shape (n_samples, ) where n_samples is the number
      of rows in the training dataset. When string is passed, it is interpreted
      as the column name in the dataset containing group labels.
  
  
  experiment_custom_tags: dict, default = None
      Dictionary of tag_name: String -> value: (String, but will be string-ified
      if not) passed to the mlflow.set_tags to add new custom tags for the experiment.
  
  
  probability_threshold: float, default = None
      Threshold for converting predicted probability to class label.
      It defaults to 0.5 for all classifiers unless explicitly defined
      in this parameter. Only applicable for binary classification.
  
  
  engine: Optional[Dict[str, str]] = None
      The execution engines to use for the models in the form of a dict
      of `model_id: engine` - e.g. for Logistic Regression ("lr", users can
      switch between "sklearn" and "sklearnex" by specifying
      `engine={"lr": "sklearnex"}`
  
  
  verbose: bool, default = True
      Score grid is not printed when verbose is set to False.
  
  
  parallel: pycaret.internal.parallel.parallel_backend.ParallelBackend, default = None
      A ParallelBackend instance. For example if you have a SparkSession ``session``,
      you can use ``FugueBackend(session)`` to make this function running using
      Spark. For more details, see
      :class:`~pycaret.parallel.fugue_backend.FugueBackend`
  
  
  Returns:
      Trained model or list of trained models, depending on the ``n_select`` param.
  
  
  Warnings
  --------
  - Changing turbo parameter to False may result in very high training times with
  datasets exceeding 10,000 rows.
  
  - AUC for estimators that does not support 'predict_proba' is shown as 0.0000.
  
  - No models are logged in ``MLFlow`` when ``cross_validation`` parameter is False.
docstr_tok: ["This","function","trains","and","evaluates","performance","of","all","estimators","available","in","the","model","library","using","cross","validation",".","The","output","of","this","function","is","a","score","grid","with","average","cross","validated","scores",".","Metrics","evaluated","during","CV","can","be","accessed","using","the","``","get_metrics","``","function",".","Custom","metrics","can","be","added","or","removed","using","``","add_metric","``","and","``","remove_metric","``","function",".","Example","--","--","--","-",">",">",">","from","pycaret.datasets","import","get_data",">",">",">","juice","=","get_data","(","'juice","'",")",">",">",">","from","pycaret.classification","import","*",">",">",">","exp_name","=","setup","(","data","=","juice",",","target","=","'Purchase","'",")",">",">",">","best_model","=","compare_models","(",")","include",":","list","of","str","or","scikit-learn","compatible","object",",","default","=","None","To","train","and","evaluate","select","models",",","list","containing","model","ID","or","scikit-learn","compatible","object","can","be","passed","in","include","param",".","To","see","a","list","of","all","models","available","in","the","model","library","use","the","``","models","``","function",".","exclude",":","list","of","str",",","default","=","None","To","omit","certain","models","from","training","and","evaluation",",","pass","a","list","containing","model","id","in","the","exclude","parameter",".","To","see","a","list","of","all","models","available","in","the","model","library","use","the","``","models","``","function",".","fold",":","int","or","scikit-learn","compatible","CV","generator",",","default","=","None","Controls","cross-validation",".","If","None",",","the","CV","generator","in","the","``","fold_strategy","``","parameter","of","the","``","setup","``","function","is","used",".","When","an","integer","is","passed",",","it","is","interpreted","as","the","'n_splits","'","parameter","of","the","CV","generator","in","the","``","setup","``","function",".","round",":","int",",","default","=","4","Number","of","decimal","places","the","metrics","in","the","score","grid","will","be","rounded","to",".","cross_validation",":","bool",",","default","=","True","When","set","to","False",",","metrics","are","evaluated","on","holdout","set",".","``","fold","``","param","is","ignored","when","cross_validation","is","set","to","False",".","sort",":","str",",","default","=","'Accuracy'","The","sort","order","of","the","score","grid",".","It","also","accepts","custom","metrics","that","are","added","through","the","``","add_metric","``","function",".","n_select",":","int",",","default","=","1","Number","of","top_n","models","to","return",".","For","example",",","to","select","top","3","models","use","n_select","=","3.","budget_time",":","int","or","float",",","default","=","None","If","not","None",",","will","terminate","execution","of","the","function","after","budget_time","minutes","have","passed","and","return","results","up","to","that","point",".","turbo",":","bool",",","default","=","True","When","set","to","True",",","it","excludes","estimators","with","longer","training","times",".","To","see","which","algorithms","are","excluded","use","the","``","models","``","function",".","errors",":","str",",","default","=","'ignore'","When","set","to","'ignore","'",",","will","skip","the","model","with","exceptions","and","continue",".","If","'raise","'",",","will","break","the","function","when","exceptions","are","raised",".","fit_kwargs",":","dict",",","default","=","{","}","(","empty","dict",")","Dictionary","of","arguments","passed","to","the","fit","method","of","the","model",".","groups",":","str","or","array-like",",","with","shape","(","n_samples",",",")",",","default","=","None","Optional","group","labels","when","'GroupKFold","'","is","used","for","the","cross","validation",".","It","takes","an","array","with","shape","(","n_samples",",",")","where","n_samples","is","the","number","of","rows","in","the","training","dataset",".","When","string","is","passed",",","it","is","interpreted","as","the","column","name","in","the","dataset","containing","group","labels",".","experiment_custom_tags",":","dict",",","default","=","None","Dictionary","of","tag_name",":","String","-",">","value",":","(","String",",","but","will","be","string-ified","if","not",")","passed","to","the","mlflow.set_tags","to","add","new","custom","tags","for","the","experiment",".","probability_threshold",":","float",",","default","=","None","Threshold","for","converting","predicted","probability","to","class","label",".","It","defaults","to","0.5","for","all","classifiers","unless","explicitly","defined","in","this","parameter",".","Only","applicable","for","binary","classification",".","engine",":","Optional","[","Dict","[","str",",","str","]","]","=","None","The","execution","engines","to","use","for","the","models","in","the","form","of","a","dict","of","`","model_id",":","engine","`","-","e.g",".","for","Logistic","Regression","(","``","lr","''",",","users","can","switch","between","``","sklearn","''","and","``","sklearnex","''","by","specifying","`","engine=","{","``","lr","''",":","``","sklearnex","''","}","`","verbose",":","bool",",","default","=","True","Score","grid","is","not","printed","when","verbose","is","set","to","False",".","parallel",":","pycaret.internal.parallel.parallel_backend.ParallelBackend",",","default","=","None","A","ParallelBackend","instance",".","For","example","if","you","have","a","SparkSession","``","session","``",",","you","can","use","``","FugueBackend","(","session",")","``","to","make","this","function","running","using","Spark",".","For","more","details",",","see",":","class",":","`","~pycaret.parallel.fugue_backend.FugueBackend","`","Returns",":","Trained","model","or","list","of","trained","models",",","depending","on","the","``","n_select","``","param",".","Warnings","--","--","--","--","-","Changing","turbo","parameter","to","False","may","result","in","very","high","training","times","with","datasets","exceeding","10,000","rows",".","-","AUC","for","estimators","that","does","not","support","'predict_proba","'","is","shown","as","0.0000",".","-","No","models","are","logged","in","``","MLFlow","``","when","``","cross_validation","``","parameter","is","False","."]
code: def compare_models(
      self,
      include: Optional[List[Union[str, Any]]] = None,
      exclude: Optional[List[str]] = None,
      fold: Optional[Union[int, Any]] = None,
      round: int = 4,
      cross_validation: bool = True,
      sort: str = "Accuracy",
      n_select: int = 1,
      budget_time: Optional[float] = None,
      turbo: bool = True,
      errors: str = "ignore",
      fit_kwargs: Optional[dict] = None,
      groups: Optional[Union[str, Any]] = None,
      experiment_custom_tags: Optional[Dict[str, Any]] = None,
      probability_threshold: Optional[float] = None,
      engine: Optional[Dict[str, str]] = None,
      verbose: bool = True,
      parallel: Optional[ParallelBackend] = None,
  ) -> Union[Any, List[Any]]:
      """
      This function trains and evaluates performance of all estimators available in the
      model library using cross validation. The output of this function is a score grid
      with average cross validated scores. Metrics evaluated during CV can be accessed
      using the ``get_metrics`` function. Custom metrics can be added or removed using
      ``add_metric`` and ``remove_metric`` function.
  
      Example
      -------
      >>> from pycaret.datasets import get_data
      >>> juice = get_data('juice')
      >>> from pycaret.classification import *
      >>> exp_name = setup(data = juice,  target = 'Purchase')
      >>> best_model = compare_models()
  
  
      include: list of str or scikit-learn compatible object, default = None
          To train and evaluate select models, list containing model ID or scikit-learn
          compatible object can be passed in include param. To see a list of all models
          available in the model library use the ``models`` function.
  
  
      exclude: list of str, default = None
          To omit certain models from training and evaluation, pass a list containing
          model id in the exclude parameter. To see a list of all models available
          in the model library use the ``models`` function.
  
  
      fold: int or scikit-learn compatible CV generator, default = None
          Controls cross-validation. If None, the CV generator in the ``fold_strategy``
          parameter of the ``setup`` function is used. When an integer is passed,
          it is interpreted as the 'n_splits' parameter of the CV generator in the
          ``setup`` function.
  
  
      round: int, default = 4
          Number of decimal places the metrics in the score grid will be rounded to.
  
  
      cross_validation: bool, default = True
          When set to False, metrics are evaluated on holdout set. ``fold`` param
          is ignored when cross_validation is set to False.
  
  
      sort: str, default = 'Accuracy'
          The sort order of the score grid. It also accepts custom metrics that are
          added through the ``add_metric`` function.
  
  
      n_select: int, default = 1
          Number of top_n models to return. For example, to select top 3 models use
          n_select = 3.
  
  
      budget_time: int or float, default = None
          If not None, will terminate execution of the function after budget_time
          minutes have passed and return results up to that point.
  
  
      turbo: bool, default = True
          When set to True, it excludes estimators with longer training times. To
          see which algorithms are excluded use the ``models`` function.
  
  
      errors: str, default = 'ignore'
          When set to 'ignore', will skip the model with exceptions and continue.
          If 'raise', will break the function when exceptions are raised.
  
  
      fit_kwargs: dict, default = {} (empty dict)
          Dictionary of arguments passed to the fit method of the model.
  
  
      groups: str or array-like, with shape (n_samples,), default = None
          Optional group labels when 'GroupKFold' is used for the cross validation.
          It takes an array with shape (n_samples, ) where n_samples is the number
          of rows in the training dataset. When string is passed, it is interpreted
          as the column name in the dataset containing group labels.
  
  
      experiment_custom_tags: dict, default = None
          Dictionary of tag_name: String -> value: (String, but will be string-ified
          if not) passed to the mlflow.set_tags to add new custom tags for the experiment.
  
  
      probability_threshold: float, default = None
          Threshold for converting predicted probability to class label.
          It defaults to 0.5 for all classifiers unless explicitly defined
          in this parameter. Only applicable for binary classification.
  
  
      engine: Optional[Dict[str, str]] = None
          The execution engines to use for the models in the form of a dict
          of `model_id: engine` - e.g. for Logistic Regression ("lr", users can
          switch between "sklearn" and "sklearnex" by specifying
          `engine={"lr": "sklearnex"}`
  
  
      verbose: bool, default = True
          Score grid is not printed when verbose is set to False.
  
  
      parallel: pycaret.internal.parallel.parallel_backend.ParallelBackend, default = None
          A ParallelBackend instance. For example if you have a SparkSession ``session``,
          you can use ``FugueBackend(session)`` to make this function running using
          Spark. For more details, see
          :class:`~pycaret.parallel.fugue_backend.FugueBackend`
  
  
      Returns:
          Trained model or list of trained models, depending on the ``n_select`` param.
  
  
      Warnings
      --------
      - Changing turbo parameter to False may result in very high training times with
      datasets exceeding 10,000 rows.
  
      - AUC for estimators that does not support 'predict_proba' is shown as 0.0000.
  
      - No models are logged in ``MLFlow`` when ``cross_validation`` parameter is False.
      """
  
      caller_params = dict(locals())
  
      # No extra code above this line
  
      if engine is not None:
          # Save current engines, then set to user specified options
          initial_model_engines = self.exp_model_engines.copy()
          for estimator, eng in engine.items():
              self._set_engine(
                  estimator=estimator, engine=eng, severity="error"
              )
  
      try:
          return_values = super().compare_models(
              include=include,
              exclude=exclude,
              fold=fold,
              round=round,
              cross_validation=cross_validation,
              sort=sort,
              n_select=n_select,
              budget_time=budget_time,
              turbo=turbo,
              errors=errors,
              fit_kwargs=fit_kwargs,
              groups=groups,
              experiment_custom_tags=experiment_custom_tags,
              verbose=verbose,
              probability_threshold=probability_threshold,
              parallel=parallel,
              caller_params=caller_params,
          )
      finally:
          if engine is not None:
              # Reset the models back to the default engines
              self._set_exp_model_engines(
                  container_default_engines=get_container_default_engines(),
                  engine=initial_model_engines,
              )
  
      return return_values
  
imports: import datetime
  import gc
  import logging
  import re
  import time
  from typing import Any, Callable, Dict, List, Optional, Tuple, Union
  from unittest.mock import patch
  import numpy
  import pandas
  import plotly.express
  import sklearn
  from joblib.memory import Memory
  from scipy.optimize import shgo
  from pycaret.containers.metrics.classification import get_all_metric_containers
  from pycaret.containers.models.classification import ALL_ALLOWED_ENGINES, get_all_model_containers, get_container_default_engines
  from pycaret.internal.display import CommonDisplay
  from pycaret.internal.logging import get_logger
  from pycaret.internal.meta_estimators import CustomProbabilityThresholdClassifier, get_estimator_from_meta_estimator
  from pycaret.internal.parallel.parallel_backend import ParallelBackend
  from pycaret.internal.pipeline import Pipeline
  from pycaret.internal.preprocess.preprocessor import Preprocessor
  from pycaret.internal.pycaret_experiment.non_ts_supervised_experiment import _NonTSSupervisedExperiment
  from pycaret.internal.validation import is_sklearn_cv_generator
  from pycaret.loggers.base_logger import BaseLogger
  from pycaret.utils.constants import DATAFRAME_LIKE, SEQUENCE_LIKE, TARGET_LIKE
  from pycaret.utils.generic import MLUsecase, get_classification_task, get_label_encoder, highlight_setup
label_desc: Use image node_id 7 for calling the ClassificationExperiment obj's underlying member method code with example usage: obj.compare_models(include, exclude, fold, round, cross_validation, sort, n_select, budget_time, turbo, errors, fit_kwargs, groups, experiment_custom_tags, probability_threshold, engine, verbose, parallel) and returns: return_values