docstr: Class to encapsulate all necessary information for identifying, calculating, and
  evaluating metrics results
  
  Parameters
  ----------
  name: String
      Identifying name of the metric. Should be unique relative to any other metric names that
      might be provided by the user
  metric_function: Callable, string, None, default=None
      If callable, should expect inputs of form (target, prediction), and return a float. If
      string, will be treated as an attribute in :mod:`sklearn.metrics`. If None, `name`
      will be treated as an attribute in :mod:`sklearn.metrics`, the value of which will be
      retrieved and used as `metric_function`
  direction: {"infer", "max", "min"}, default="infer"
      How to compare the result of `metric_function` relative to previous evaluations
  
      * "max": Metric values should be maximized, and higher metric values are better than
        lower values; it should be used for measures of accuracy
      * "min": Metric values should be minimized, and lower metric values are better than
        higher values; it should be used for measures of error or loss
      * "infer": `direction` will be set to:
  
          1. "min" if `name` (or `metric_function`'s name) contains "error" or "loss"
          2. "max" if `name` contains neither of the aforementioned strings
  
  Notes
  -----
  `direction` = "infer" looks for "error"/"loss" in `name` first, then in the name of
  `metric_function`. This means that `name` can be an abbreviation/anything for error
  measures and `direction` will still be correctly inferred as long as the actual callable
  for `metric_function` has "error"/"loss" in its name. For example, `direction` = "min" is
  safely inferred when using "mae" for "mean_absolute_error" or "rmsle" for
  "root_mean_squared_logarithmic_error". This functions as described whether `metric_function`
  is a string naming an SKLearn metric, or a callable whose name includes "error"/"loss"
  
  Examples
  --------
  >>> Metric("roc_auc_score")  # doctest: +ELLIPSIS
  Metric(roc_auc_score, <function roc_auc_score at 0x...>, max)
  >>> Metric("roc_auc_score", sk_metrics.roc_auc_score)  # doctest: +ELLIPSIS
  Metric(roc_auc_score, <function roc_auc_score at 0x...>, max)
  >>> Metric("my_f1_score", "f1_score")  # doctest: +ELLIPSIS
  Metric(my_f1_score, <function f1_score at 0x...>, max)
  >>> Metric("hamming_loss", sk_metrics.hamming_loss)  # doctest: +ELLIPSIS
  Metric(hamming_loss, <function hamming_loss at 0x...>, min)
  
  *Respect explicit `direction` even if it doesn't make sense for the `metric_function`*
  
  >>> Metric("r2_score", sk_metrics.r2_score, direction="min")  # doctest: +ELLIPSIS
  Metric(r2_score, <function r2_score at 0x...>, min)
  
  *Direction inference based on `metric_function` name, rather than `name` itself*
  
  >>> Metric("mae", "median_absolute_error")  # doctest: +ELLIPSIS
  Metric(mae, <function median_absolute_error at 0x...>, min)
  >>> Metric("hl", sk_metrics.hamming_loss)  # doctest: +ELLIPSIS
  Metric(hl, <function hamming_loss at 0x...>, min)
docstr_tok: ["Class","to","encapsulate","all","necessary","information","for","identifying",",","calculating",",","and","evaluating","metrics","results","Parameters","--","--","--","--","--","name",":","String","Identifying","name","of","the","metric",".","Should","be","unique","relative","to","any","other","metric","names","that","might","be","provided","by","the","user","metric_function",":","Callable",",","string",",","None",",","default=None","If","callable",",","should","expect","inputs","of","form","(","target",",","prediction",")",",","and","return","a","float",".","If","string",",","will","be","treated","as","an","attribute","in",":","mod",":","`","sklearn.metrics","`",".","If","None",",","`","name","`","will","be","treated","as","an","attribute","in",":","mod",":","`","sklearn.metrics","`",",","the","value","of","which","will","be","retrieved","and","used","as","`","metric_function","`","direction",":","{","``","infer","''",",","``","max","''",",","``","min","''","}",",","default=","''","infer","''","How","to","compare","the","result","of","`","metric_function","`","relative","to","previous","evaluations","*","``","max","''",":","Metric","values","should","be","maximized",",","and","higher","metric","values","are","better","than","lower","values",";","it","should","be","used","for","measures","of","accuracy","*","``","min","''",":","Metric","values","should","be","minimized",",","and","lower","metric","values","are","better","than","higher","values",";","it","should","be","used","for","measures","of","error","or","loss","*","``","infer","''",":","`","direction","`","will","be","set","to",":","1",".","``","min","''","if","`","name","`","(","or","`","metric_function","`","'s","name",")","contains","``","error","''","or","``","loss","''","2",".","``","max","''","if","`","name","`","contains","neither","of","the","aforementioned","strings","Notes","--","--","-","`","direction","`","=","``","infer","''","looks","for","``","error","''","\/","''","loss","''","in","`","name","`","first",",","then","in","the","name","of","`","metric_function","`",".","This","means","that","`","name","`","can","be","an","abbreviation\/anything","for","error","measures","and","`","direction","`","will","still","be","correctly","inferred","as","long","as","the","actual","callable","for","`","metric_function","`","has","``","error","''","\/","''","loss","''","in","its","name",".","For","example",",","`","direction","`","=","``","min","''","is","safely","inferred","when","using","``","mae","''","for","``","mean_absolute_error","''","or","``","rmsle","''","for","''","root_mean_squared_logarithmic_error","''",".","This","functions","as","described","whether","`","metric_function","`","is","a","string","naming","an","SKLearn","metric",",","or","a","callable","whose","name","includes","``","error","''","\/","''","loss","''","Examples","--","--","--","--",">",">",">","Metric","(","``","roc_auc_score","''",")","#","doctest",":","+ELLIPSIS","Metric","(","roc_auc_score",",","<","function","roc_auc_score","at","0x","...",">",",","max",")",">",">",">","Metric","(","``","roc_auc_score","''",",","sk_metrics.roc_auc_score",")","#","doctest",":","+ELLIPSIS","Metric","(","roc_auc_score",",","<","function","roc_auc_score","at","0x","...",">",",","max",")",">",">",">","Metric","(","``","my_f1_score","''",",","``","f1_score","''",")","#","doctest",":","+ELLIPSIS","Metric","(","my_f1_score",",","<","function","f1_score","at","0x","...",">",",","max",")",">",">",">","Metric","(","``","hamming_loss","''",",","sk_metrics.hamming_loss",")","#","doctest",":","+ELLIPSIS","Metric","(","hamming_loss",",","<","function","hamming_loss","at","0x","...",">",",","min",")","*","Respect","explicit","`","direction","`","even","if","it","does","n't","make","sense","for","the","`","metric_function","`","*",">",">",">","Metric","(","``","r2_score","''",",","sk_metrics.r2_score",",","direction=","''","min","''",")","#","doctest",":","+ELLIPSIS","Metric","(","r2_score",",","<","function","r2_score","at","0x","...",">",",","min",")","*","Direction","inference","based","on","`","metric_function","`","name",",","rather","than","`","name","`","itself","*",">",">",">","Metric","(","``","mae","''",",","``","median_absolute_error","''",")","#","doctest",":","+ELLIPSIS","Metric","(","mae",",","<","function","median_absolute_error","at","0x","...",">",",","min",")",">",">",">","Metric","(","``","hl","''",",","sk_metrics.hamming_loss",")","#","doctest",":","+ELLIPSIS","Metric","(","hl",",","<","function","hamming_loss","at","0x","...",">",",","min",")"]
code: def __init__(
      self,
      name: str,
      metric_function: Union[callable, str, None] = None,
      direction: str = "infer",
  ):
      """Class to encapsulate all necessary information for identifying, calculating, and
      evaluating metrics results
  
      Parameters
      ----------
      name: String
          Identifying name of the metric. Should be unique relative to any other metric names that
          might be provided by the user
      metric_function: Callable, string, None, default=None
          If callable, should expect inputs of form (target, prediction), and return a float. If
          string, will be treated as an attribute in :mod:`sklearn.metrics`. If None, `name`
          will be treated as an attribute in :mod:`sklearn.metrics`, the value of which will be
          retrieved and used as `metric_function`
      direction: {"infer", "max", "min"}, default="infer"
          How to compare the result of `metric_function` relative to previous evaluations
  
          * "max": Metric values should be maximized, and higher metric values are better than
            lower values; it should be used for measures of accuracy
          * "min": Metric values should be minimized, and lower metric values are better than
            higher values; it should be used for measures of error or loss
          * "infer": `direction` will be set to:
  
              1. "min" if `name` (or `metric_function`'s name) contains "error" or "loss"
              2. "max" if `name` contains neither of the aforementioned strings
  
      Notes
      -----
      `direction` = "infer" looks for "error"/"loss" in `name` first, then in the name of
      `metric_function`. This means that `name` can be an abbreviation/anything for error
      measures and `direction` will still be correctly inferred as long as the actual callable
      for `metric_function` has "error"/"loss" in its name. For example, `direction` = "min" is
      safely inferred when using "mae" for "mean_absolute_error" or "rmsle" for
      "root_mean_squared_logarithmic_error". This functions as described whether `metric_function`
      is a string naming an SKLearn metric, or a callable whose name includes "error"/"loss"
  
      Examples
      --------
      >>> Metric("roc_auc_score")  # doctest: +ELLIPSIS
      Metric(roc_auc_score, <function roc_auc_score at 0x...>, max)
      >>> Metric("roc_auc_score", sk_metrics.roc_auc_score)  # doctest: +ELLIPSIS
      Metric(roc_auc_score, <function roc_auc_score at 0x...>, max)
      >>> Metric("my_f1_score", "f1_score")  # doctest: +ELLIPSIS
      Metric(my_f1_score, <function f1_score at 0x...>, max)
      >>> Metric("hamming_loss", sk_metrics.hamming_loss)  # doctest: +ELLIPSIS
      Metric(hamming_loss, <function hamming_loss at 0x...>, min)
  
      *Respect explicit `direction` even if it doesn't make sense for the `metric_function`*
  
      >>> Metric("r2_score", sk_metrics.r2_score, direction="min")  # doctest: +ELLIPSIS
      Metric(r2_score, <function r2_score at 0x...>, min)
  
      *Direction inference based on `metric_function` name, rather than `name` itself*
  
      >>> Metric("mae", "median_absolute_error")  # doctest: +ELLIPSIS
      Metric(mae, <function median_absolute_error at 0x...>, min)
      >>> Metric("hl", sk_metrics.hamming_loss)  # doctest: +ELLIPSIS
      Metric(hl, <function hamming_loss at 0x...>, min)"""
      self.name = name
      self.metric_function = self._set_metric_function(metric_function)
      self.direction = self._set_direction(direction)
  
imports: from collections import OrderedDict
  from contextlib import suppress
  import numpy
  import pandas
  from typing import Dict, Iterable, List, Tuple, Union
  from sklearn import metrics
  from sklearn.utils.multiclass import type_of_target, unique_labels
label_desc: Use image node_id 1 to create a new Metric object from inherited base classes: object with example: obj = Metric(name, metric_function, direction)