class_docstr: AutoMM is designed to simplify the fine-tuning of foundation models
  for downstream applications with just three lines of code.
  AutoMM seamlessly integrates with popular model zoos such as
  `HuggingFace Transformers <https://github.com/huggingface/transformers>`_,
  `TIMM <https://github.com/huggingface/pytorch-image-models>`_,
  and `MMDetection <https://github.com/open-mmlab/mmdetection>`_,
  accommodating a diverse range of data modalities,
  including image, text, tabular, and document data, whether used individually or in combination.
  It offers support for an array of tasks, encompassing classification, regression,
  object detection, named entity recognition, semantic matching, and image segmentation.
class_docstr_tok: ["AutoMM","is","designed","to","simplify","the","fine-tuning","of","foundation","models","for","downstream","applications","with","just","three","lines","of","code",".","AutoMM","seamlessly","integrates","with","popular","model","zoos","such","as","`","HuggingFace","Transformers","<","https",":","\/\/github.com\/huggingface\/transformers",">","`","_",",","`","TIMM","<","https",":","\/\/github.com\/huggingface\/pytorch-image-models",">","`","_",",","and","`","MMDetection","<","https",":","\/\/github.com\/open-mmlab\/mmdetection",">","`","_",",","accommodating","a","diverse","range","of","data","modalities",",","including","image",",","text",",","tabular",",","and","document","data",",","whether","used","individually","or","in","combination",".","It","offers","support","for","an","array","of","tasks",",","encompassing","classification",",","regression",",","object","detection",",","named","entity","recognition",",","semantic","matching",",","and","image","segmentation","."]
docstr: Parameters
  ----------
  label
      Name of one pd.DataFrame column that contains the target variable to predict.
  problem_type
      Type of problem. We support standard problems like
  
      - 'binary': Binary classification
      - 'multiclass': Multi-class classification
      - 'regression': Regression
      - 'classification': Classification problems include 'binary' and 'multiclass' classification.
  
      In addition, we support advanced problems such as
  
      - 'object_detection': Object detection
      - 'open_vocabulry_object_detection': Zero-shot object detection (only support inference)
      - 'ner' or 'named_entity_recognition': Named entity extraction
      - 'text_similarity': Text-text semantic matching
      - 'image_similarity': Image-image semantic matching
      - 'image_text_similarity': Text-image semantic matching
      - 'feature_extraction': Extracting feature (only support inference)
      - 'zero_shot_image_classification': Zero-shot image classification (only support inference)
      - 'few_shot_classification': Few-shot classification for image or text data.
      - 'semantic_segmentation': Semantic segmentation with Segment Anything Model.
  
      For certain problem types, the default behavior is to load a pretrained model based on
      the presets / hyperparameters and the predictor can do zero-shot inference
      (running inference without .fit()). Those include the following
      problem types:
  
      - 'object_detection'
      - 'open_vocabulary_object_detection'
      - 'text_similarity'
      - 'image_similarity'
      - 'image_text_similarity'
      - 'feature_extraction'
      - 'zero_shot_image_classification'
  
  query
      Name of one pd.DataFrame column that has the query data in semantic matching tasks.
  response
      Name of one pd.DataFrame column that contains the response data in semantic matching tasks.
      If no label column is provided, the query and response pairs in
      one pd.DataFrame row are assumed to be positive pairs.
  match_label
      The label class that indicates the <query, response> pair is counted as a "match".
      This is used when the task belongs to semantic matching, and the labels are binary.
      For example, the label column can contain ["duplicate", "not duplicate"] in a duplicate detection task.
      The match_label should be "duplicate" since it means that two items match.
  presets
      Presets regarding model quality, e.g., 'best_quality', 'high_quality' (default), and 'medium_quality'.
      Each quality has its corresponding HPO presets: 'best_quality_hpo', 'high_quality_hpo', and 'medium_quality_hpo'.
  eval_metric
      Evaluation metric name. If `eval_metric = None`, it is automatically chosen based on `problem_type`.
      Defaults to 'accuracy' for multiclass classification, `roc_auc` for binary classification,
      and 'root_mean_squared_error' for regression.
  hyperparameters
      This is to override some default configurations.
      For example, changing the text and image backbones can be done by formatting:
  
      a string
      hyperparameters = "model.hf_text.checkpoint_name=google/electra-small-discriminator model.timm_image.checkpoint_name=swin_small_patch4_window7_224"
  
      or a list of strings
      hyperparameters = ["model.hf_text.checkpoint_name=google/electra-small-discriminator", "model.timm_image.checkpoint_name=swin_small_patch4_window7_224"]
  
      or a dictionary
      hyperparameters = {
                      "model.hf_text.checkpoint_name": "google/electra-small-discriminator",
                      "model.timm_image.checkpoint_name": "swin_small_patch4_window7_224",
                  }
  path
      Path to directory where models and related artifacts should be saved.
      If unspecified, a time-stamped folder called "AutogluonAutoMM/ag-[TIMESTAMP]"
      will be created in the working directory.
      Note: To call `fit()` twice and save all results of each fit,
      you must specify different `path` locations or don't specify `path` at all.
  verbosity
      Verbosity levels range from 0 to 4, controlling how much logging information is printed.
      Higher levels correspond to more detailed print statements.
      You can set verbosity = 0 to suppress warnings.
  num_classes
      Number of classes (used for object detection).
      If this is specified and is different from the pretrained model's output shape,
      the model's head will be changed to have <num_classes> output.
  classes
      All the classes (used for object detection).
  warn_if_exist
      Whether to raise warning if the specified path already exists (Default True).
  enable_progress_bar
      Whether to show progress bar (default True). It would be
      disabled if the environment variable os.environ["AUTOMM_DISABLE_PROGRESS_BAR"] is set.
  pretrained
      Whether to initialize the model with pretrained weights (default True).
      If False, it creates a model with random initialization.
  validation_metric
      Validation metric for selecting the best model and early-stopping during training.
      If not provided, it would be automatically chosen based on the problem type.
  sample_data_path
      The path to sample data from which we can infer num_classes or classes used for object detection.
docstr_tok: ["Parameters","--","--","--","--","--","label","Name","of","one","pd.DataFrame","column","that","contains","the","target","variable","to","predict",".","problem_type","Type","of","problem",".","We","support","standard","problems","like","-","'binary","'",":","Binary","classification","-","'multiclass","'",":","Multi-class","classification","-","'regression","'",":","Regression","-","'classification","'",":","Classification","problems","include","'binary","'","and","'multiclass","'","classification",".","In","addition",",","we","support","advanced","problems","such","as","-","'object_detection","'",":","Object","detection","-","'open_vocabulry_object_detection","'",":","Zero-shot","object","detection","(","only","support","inference",")","-","'ner","'","or","'named_entity_recognition","'",":","Named","entity","extraction","-","'text_similarity","'",":","Text-text","semantic","matching","-","'image_similarity","'",":","Image-image","semantic","matching","-","'image_text_similarity","'",":","Text-image","semantic","matching","-","'feature_extraction","'",":","Extracting","feature","(","only","support","inference",")","-","'zero_shot_image_classification","'",":","Zero-shot","image","classification","(","only","support","inference",")","-","'few_shot_classification","'",":","Few-shot","classification","for","image","or","text","data",".","-","'semantic_segmentation","'",":","Semantic","segmentation","with","Segment","Anything","Model",".","For","certain","problem","types",",","the","default","behavior","is","to","load","a","pretrained","model","based","on","the","presets","\/","hyperparameters","and","the","predictor","can","do","zero-shot","inference","(","running","inference","without",".fit","(",")",")",".","Those","include","the","following","problem","types",":","-","'object_detection'","-","'open_vocabulary_object_detection'","-","'text_similarity'","-","'image_similarity'","-","'image_text_similarity'","-","'feature_extraction'","-","'zero_shot_image_classification'","query","Name","of","one","pd.DataFrame","column","that","has","the","query","data","in","semantic","matching","tasks",".","response","Name","of","one","pd.DataFrame","column","that","contains","the","response","data","in","semantic","matching","tasks",".","If","no","label","column","is","provided",",","the","query","and","response","pairs","in","one","pd.DataFrame","row","are","assumed","to","be","positive","pairs",".","match_label","The","label","class","that","indicates","the","<","query",",","response",">","pair","is","counted","as","a","``","match","''",".","This","is","used","when","the","task","belongs","to","semantic","matching",",","and","the","labels","are","binary",".","For","example",",","the","label","column","can","contain","[","``","duplicate","''",",","``","not","duplicate","''","]","in","a","duplicate","detection","task",".","The","match_label","should","be","``","duplicate","''","since","it","means","that","two","items","match",".","presets","Presets","regarding","model","quality",",","e.g.",",","'best_quality","'",",","'high_quality","'","(","default",")",",","and","'medium_quality","'",".","Each","quality","has","its","corresponding","HPO","presets",":","'best_quality_hpo","'",",","'high_quality_hpo","'",",","and","'medium_quality_hpo","'",".","eval_metric","Evaluation","metric","name",".","If","`","eval_metric","=","None","`",",","it","is","automatically","chosen","based","on","`","problem_type","`",".","Defaults","to","'accuracy","'","for","multiclass","classification",",","`","roc_auc","`","for","binary","classification",",","and","'root_mean_squared_error","'","for","regression",".","hyperparameters","This","is","to","override","some","default","configurations",".","For","example",",","changing","the","text","and","image","backbones","can","be","done","by","formatting",":","a","string","hyperparameters","=","``","model.hf_text.checkpoint_name=google\/electra-small-discriminator","model.timm_image.checkpoint_name=swin_small_patch4_window7_224","''","or","a","list","of","strings","hyperparameters","=","[","``","model.hf_text.checkpoint_name=google\/electra-small-discriminator","''",",","``","model.timm_image.checkpoint_name=swin_small_patch4_window7_224","''","]","or","a","dictionary","hyperparameters","=","{","``","model.hf_text.checkpoint_name","''",":","``","google\/electra-small-discriminator","''",",","``","model.timm_image.checkpoint_name","''",":","``","swin_small_patch4_window7_224","''",",","}","path","Path","to","directory","where","models","and","related","artifacts","should","be","saved",".","If","unspecified",",","a","time-stamped","folder","called","``","AutogluonAutoMM\/ag-","[","TIMESTAMP","]","''","will","be","created","in","the","working","directory",".","Note",":","To","call","`","fit","(",")","`","twice","and","save","all","results","of","each","fit",",","you","must","specify","different","`","path","`","locations","or","do","n't","specify","`","path","`","at","all",".","verbosity","Verbosity","levels","range","from","0","to","4",",","controlling","how","much","logging","information","is","printed",".","Higher","levels","correspond","to","more","detailed","print","statements",".","You","can","set","verbosity","=","0","to","suppress","warnings",".","num_classes","Number","of","classes","(","used","for","object","detection",")",".","If","this","is","specified","and","is","different","from","the","pretrained","model","'s","output","shape",",","the","model","'s","head","will","be","changed","to","have","<","num_classes",">","output",".","classes","All","the","classes","(","used","for","object","detection",")",".","warn_if_exist","Whether","to","raise","warning","if","the","specified","path","already","exists","(","Default","True",")",".","enable_progress_bar","Whether","to","show","progress","bar","(","default","True",")",".","It","would","be","disabled","if","the","environment","variable","os.environ","[","``","AUTOMM_DISABLE_PROGRESS_BAR","''","]","is","set",".","pretrained","Whether","to","initialize","the","model","with","pretrained","weights","(","default","True",")",".","If","False",",","it","creates","a","model","with","random","initialization",".","validation_metric","Validation","metric","for","selecting","the","best","model","and","early-stopping","during","training",".","If","not","provided",",","it","would","be","automatically","chosen","based","on","the","problem","type",".","sample_data_path","The","path","to","sample","data","from","which","we","can","infer","num_classes","or","classes","used","for","object","detection","."]
code: def __init__(
      self,
      label: Optional[str] = None,
      problem_type: Optional[str] = None,
      query: Optional[Union[str, List[str]]] = None,
      response: Optional[Union[str, List[str]]] = None,
      match_label: Optional[Union[int, str]] = None,
      presets: Optional[str] = None,
      eval_metric: Optional[Union[str, Scorer]] = None,
      hyperparameters: Optional[dict] = None,
      path: Optional[str] = None,
      verbosity: Optional[int] = 2,
      num_classes: Optional[
          int
      ] = None,  # TODO: can we infer this from data?
      classes: Optional[list] = None,
      warn_if_exist: Optional[bool] = True,
      enable_progress_bar: Optional[bool] = None,
      pretrained: Optional[bool] = True,
      validation_metric: Optional[str] = None,
      sample_data_path: Optional[str] = None,
  ):
      """
      Parameters
      ----------
      label
          Name of one pd.DataFrame column that contains the target variable to predict.
      problem_type
          Type of problem. We support standard problems like
  
          - 'binary': Binary classification
          - 'multiclass': Multi-class classification
          - 'regression': Regression
          - 'classification': Classification problems include 'binary' and 'multiclass' classification.
  
          In addition, we support advanced problems such as
  
          - 'object_detection': Object detection
          - 'open_vocabulry_object_detection': Zero-shot object detection (only support inference)
          - 'ner' or 'named_entity_recognition': Named entity extraction
          - 'text_similarity': Text-text semantic matching
          - 'image_similarity': Image-image semantic matching
          - 'image_text_similarity': Text-image semantic matching
          - 'feature_extraction': Extracting feature (only support inference)
          - 'zero_shot_image_classification': Zero-shot image classification (only support inference)
          - 'few_shot_classification': Few-shot classification for image or text data.
          - 'semantic_segmentation': Semantic segmentation with Segment Anything Model.
  
          For certain problem types, the default behavior is to load a pretrained model based on
          the presets / hyperparameters and the predictor can do zero-shot inference
          (running inference without .fit()). Those include the following
          problem types:
  
          - 'object_detection'
          - 'open_vocabulary_object_detection'
          - 'text_similarity'
          - 'image_similarity'
          - 'image_text_similarity'
          - 'feature_extraction'
          - 'zero_shot_image_classification'
  
      query
          Name of one pd.DataFrame column that has the query data in semantic matching tasks.
      response
          Name of one pd.DataFrame column that contains the response data in semantic matching tasks.
          If no label column is provided, the query and response pairs in
          one pd.DataFrame row are assumed to be positive pairs.
      match_label
          The label class that indicates the <query, response> pair is counted as a "match".
          This is used when the task belongs to semantic matching, and the labels are binary.
          For example, the label column can contain ["duplicate", "not duplicate"] in a duplicate detection task.
          The match_label should be "duplicate" since it means that two items match.
      presets
          Presets regarding model quality, e.g., 'best_quality', 'high_quality' (default), and 'medium_quality'.
          Each quality has its corresponding HPO presets: 'best_quality_hpo', 'high_quality_hpo', and 'medium_quality_hpo'.
      eval_metric
          Evaluation metric name. If `eval_metric = None`, it is automatically chosen based on `problem_type`.
          Defaults to 'accuracy' for multiclass classification, `roc_auc` for binary classification,
          and 'root_mean_squared_error' for regression.
      hyperparameters
          This is to override some default configurations.
          For example, changing the text and image backbones can be done by formatting:
  
          a string
          hyperparameters = "model.hf_text.checkpoint_name=google/electra-small-discriminator model.timm_image.checkpoint_name=swin_small_patch4_window7_224"
  
          or a list of strings
          hyperparameters = ["model.hf_text.checkpoint_name=google/electra-small-discriminator", "model.timm_image.checkpoint_name=swin_small_patch4_window7_224"]
  
          or a dictionary
          hyperparameters = {
                          "model.hf_text.checkpoint_name": "google/electra-small-discriminator",
                          "model.timm_image.checkpoint_name": "swin_small_patch4_window7_224",
                      }
      path
          Path to directory where models and related artifacts should be saved.
          If unspecified, a time-stamped folder called "AutogluonAutoMM/ag-[TIMESTAMP]"
          will be created in the working directory.
          Note: To call `fit()` twice and save all results of each fit,
          you must specify different `path` locations or don't specify `path` at all.
      verbosity
          Verbosity levels range from 0 to 4, controlling how much logging information is printed.
          Higher levels correspond to more detailed print statements.
          You can set verbosity = 0 to suppress warnings.
      num_classes
          Number of classes (used for object detection).
          If this is specified and is different from the pretrained model's output shape,
          the model's head will be changed to have <num_classes> output.
      classes
          All the classes (used for object detection).
      warn_if_exist
          Whether to raise warning if the specified path already exists (Default True).
      enable_progress_bar
          Whether to show progress bar (default True). It would be
          disabled if the environment variable os.environ["AUTOMM_DISABLE_PROGRESS_BAR"] is set.
      pretrained
          Whether to initialize the model with pretrained weights (default True).
          If False, it creates a model with random initialization.
      validation_metric
          Validation metric for selecting the best model and early-stopping during training.
          If not provided, it would be automatically chosen based on the problem type.
      sample_data_path
          The path to sample data from which we can infer num_classes or classes used for object detection.
      """
      if problem_type is not None:
          problem_type = problem_type.lower()
          assert problem_type in PROBLEM_TYPES_REG, (
              f"problem_type='{problem_type}' is not supported yet. You may pick a problem type from"
              f" {PROBLEM_TYPES_REG.list_keys()}."
          )
          problem_property = PROBLEM_TYPES_REG.get(problem_type)
          if problem_property.experimental:
              warnings.warn(
                  f"problem_type='{problem_type}' is currently experimental.",
                  UserWarning,
              )
          problem_type = problem_property.name
      else:
          problem_property = None
  
      if os.environ.get(AUTOMM_TUTORIAL_MODE):
          enable_progress_bar = False
          # Also disable progress bar of transformers package
          transformers.logging.disable_progress_bar()
  
      if verbosity is not None:
          set_logger_verbosity(verbosity)
  
      self._verbosity = verbosity
  
      if problem_property and problem_property.is_matching:
          learner_class = MultiModalMatcher
      elif problem_type in [
          OBJECT_DETECTION,
          OPEN_VOCABULARY_OBJECT_DETECTION,
      ]:
          learner_class = ObjectDetectionLearner
      elif problem_type == NER:
          learner_class = NERLearner
      elif problem_type == FEW_SHOT_CLASSIFICATION:
          learner_class = FewShotSVMLearner
      elif problem_type == SEMANTIC_SEGMENTATION:
          learner_class = SemanticSegmentationLearner
      else:
          learner_class = BaseLearner
  
      self._learner = learner_class(
          label=label,
          problem_type=problem_type,
          presets=presets,
          eval_metric=eval_metric,
          hyperparameters=hyperparameters,
          path=path,
          verbosity=verbosity,
          num_classes=num_classes,
          classes=classes,
          warn_if_exist=warn_if_exist,
          enable_progress_bar=enable_progress_bar,
          pretrained=pretrained,
          sample_data_path=sample_data_path,
          validation_metric=validation_metric,
          query=query,
          response=response,
          match_label=match_label,
      )
  
imports: from __future__ import annotations
  import json
  import logging
  import os
  import warnings
  from typing import Dict, List, Optional, Union
  import pandas
  import transformers
  from autogluon.common.utils.log_utils import set_logger_verbosity, verbosity2loglevel
  from autogluon.core.metrics import Scorer
  from .constants import AUTOMM_TUTORIAL_MODE, FEW_SHOT_CLASSIFICATION, NER, OBJECT_DETECTION, OPEN_VOCABULARY_OBJECT_DETECTION, SEMANTIC_SEGMENTATION
  from .learners import BaseLearner, FewShotSVMLearner, MultiModalMatcher, NERLearner, ObjectDetectionLearner, SemanticSegmentationLearner
  from .problem_types import PROBLEM_TYPES_REG
  from .utils import get_dir_ckpt_paths
label_desc: Use image node_id 1 to create a new MultiModalPredictor object  with example: obj = MultiModalPredictor(label, problem_type, query, response, match_label, presets, eval_metric, hyperparameters, path, verbosity, num_classes, classes, warn_if_exist, enable_progress_bar, pretrained, validation_metric, sample_data_path)