class_docstr: The class object for computing MDI+ feature importances for a forest or collection of trees.
  Generalized mean decrease in impurity (MDI+) is a flexible framework for computing RF
  feature importances. For more details, refer to [paper].
  
  Parameters
  ----------
  estimators: list of fitted PartialPredictionModelBase objects or scikit-learn type estimators
      The fitted partial prediction models (one per tree) to use for evaluating
      feature importance via MDI+. If not a PartialPredictionModelBase, then
      the estimator is coerced into a PartialPredictionModelBase object via
      GenericRegressorPPM or GenericClassifierPPM depending on the specified
      task. Note that these generic PPMs may be computationally expensive.
  transformers: list of BlockTransformerBase objects
      The block feature transformers used to generate blocks of engineered
      features for each original feature. The transformed data is then used
      as input into the partial prediction models. Should be the same length
      as estimators.
  scoring_fns: a function or dict with functions as value and function name (str) as key
      The scoring functions used for evaluating the partial predictions.
  sample_split: string in {"loo", "oob", "inbag"} or None
      The sample splitting strategy to be used when evaluating the partial
      model predictions. The default "loo" (leave-one-out) is strongly
      recommended for performance and in particular, for overcoming the known
      correlation and entropy biases suffered by MDI. "oob" (out-of-bag) can
      also be used to overcome these biases. "inbag" is the sample splitting
      strategy used by MDI. If None, no sample splitting is performed and the
      full data set is used to evaluate the partial model predictions.
  tree_random_states: list of int or None
      Random states from each tree in the fitted random forest; used in
      sample splitting and only required if sample_split = "oob" or "inbag".
      Should be the same length as estimators.
  mode: string in {"keep_k", "keep_rest"}
      Mode for the method. "keep_k" imputes the mean of each feature not
      in block k when making a partial model prediction, while "keep_rest"
      imputes the mean of each feature in block k. "keep_k" is strongly
      recommended for computational considerations.
  task: string in {"regression", "classification"}
      The supervised learning task for the RF model. Used for choosing
      defaults for the scoring_fns. Currently only regression and
      classification are supported.
  center: bool
      Flag for whether to center the transformed data in the transformers.
  normalize: bool
      Flag for whether to rescale the transformed data to have unit
      variance in the transformers.
docstr: Evaluate the stability of the MDI+ feature importance rankings
  across bootstrapped samples of trees. Can be used to select the GLM
  and scoring metric in a data-driven manner, where the GLM and metric that
  yields the most stable feature rankings across bootstrapped samples is selected.
  
  Parameters
  ----------
  B: int
      Number of bootstrap samples.
  metrics: "auto" or a dict with functions as value and function name (str) as key
      Metric(s) used to evaluate the stability between two sets of feature importances.
      If "auto", then the feature importance stability metrics are:
          (1) Rank-based overlap (RBO) with p=0.9 (from "A Similarity Measure for
          Indefinite Rankings" by Webber et al. (2010)). Intuitively, this metric gives
          more weight to features with the largest importances, with most of the weight
          going to the ~1/(1-p) features with the largest importances.
          (2) A weighted kendall tau metric (tauAP_b from "The Treatment of Ties in
          AP Correlation" by Urbano and Marrero (2017)), which also gives more weight
          to the features with the largest importances, but uses a different weighting
          scheme from RBO.
      Note that these default metrics assume that a higher MDI+ score indicates
      greater importance and thus give more weight to these features with high
      importance/ranks. If a lower MDI+ score indicates higher importance, then invert
      either these stability metrics or the MDI+ scores before evaluating the stability.
  
  Returns
  -------
  stability_results: pd.DataFrame of shape (n_features, n_metrics)
      The stability scores of the MDI+ feature rankings across bootstrapped samples.
docstr_tok: ["Evaluate","the","stability","of","the","MDI+","feature","importance","rankings","across","bootstrapped","samples","of","trees",".","Can","be","used","to","select","the","GLM","and","scoring","metric","in","a","data-driven","manner",",","where","the","GLM","and","metric","that","yields","the","most","stable","feature","rankings","across","bootstrapped","samples","is","selected",".","Parameters","--","--","--","--","--","B",":","int","Number","of","bootstrap","samples",".","metrics",":","``","auto","''","or","a","dict","with","functions","as","value","and","function","name","(","str",")","as","key","Metric","(","s",")","used","to","evaluate","the","stability","between","two","sets","of","feature","importances",".","If","``","auto","''",",","then","the","feature","importance","stability","metrics","are",":","(","1",")","Rank-based","overlap","(","RBO",")","with","p=0.9","(","from","``","A","Similarity","Measure","for","Indefinite","Rankings","''","by","Webber","et","al",".","(","2010",")",")",".","Intuitively",",","this","metric","gives","more","weight","to","features","with","the","largest","importances",",","with","most","of","the","weight","going","to","the","~1\/","(","1-p",")","features","with","the","largest","importances",".","(","2",")","A","weighted","kendall","tau","metric","(","tauAP_b","from","``","The","Treatment","of","Ties","in","AP","Correlation","''","by","Urbano","and","Marrero","(","2017",")",")",",","which","also","gives","more","weight","to","the","features","with","the","largest","importances",",","but","uses","a","different","weighting","scheme","from","RBO",".","Note","that","these","default","metrics","assume","that","a","higher","MDI+","score","indicates","greater","importance","and","thus","give","more","weight","to","these","features","with","high","importance\/ranks",".","If","a","lower","MDI+","score","indicates","higher","importance",",","then","invert","either","these","stability","metrics","or","the","MDI+","scores","before","evaluating","the","stability",".","Returns","--","--","--","-","stability_results",":","pd.DataFrame","of","shape","(","n_features",",","n_metrics",")","The","stability","scores","of","the","MDI+","feature","rankings","across","bootstrapped","samples","."]
code: def get_stability_scores(self, B=10, metrics="auto"):
      """
      Evaluate the stability of the MDI+ feature importance rankings
      across bootstrapped samples of trees. Can be used to select the GLM
      and scoring metric in a data-driven manner, where the GLM and metric that
      yields the most stable feature rankings across bootstrapped samples is selected.
  
      Parameters
      ----------
      B: int
          Number of bootstrap samples.
      metrics: "auto" or a dict with functions as value and function name (str) as key
          Metric(s) used to evaluate the stability between two sets of feature importances.
          If "auto", then the feature importance stability metrics are:
              (1) Rank-based overlap (RBO) with p=0.9 (from "A Similarity Measure for
              Indefinite Rankings" by Webber et al. (2010)). Intuitively, this metric gives
              more weight to features with the largest importances, with most of the weight
              going to the ~1/(1-p) features with the largest importances.
              (2) A weighted kendall tau metric (tauAP_b from "The Treatment of Ties in
              AP Correlation" by Urbano and Marrero (2017)), which also gives more weight
              to the features with the largest importances, but uses a different weighting
              scheme from RBO.
          Note that these default metrics assume that a higher MDI+ score indicates
          greater importance and thus give more weight to these features with high
          importance/ranks. If a lower MDI+ score indicates higher importance, then invert
          either these stability metrics or the MDI+ scores before evaluating the stability.
  
      Returns
      -------
      stability_results: pd.DataFrame of shape (n_features, n_metrics)
          The stability scores of the MDI+ feature rankings across bootstrapped samples.
  
      """
      if metrics == "auto":
          metrics = {"RBO": partial(rbo, p=0.9), "tauAP": tauAP_b}
      elif not isinstance(metrics, dict):
          raise ValueError(
              "`metrics` must be 'auto' or a dictionary "
              "where the key is the metric name and the value is the evaluation function"
          )
      single_scoring_fn = not isinstance(
          self.feature_importances_by_tree_, dict
      )
      if single_scoring_fn:
          feature_importances_dict = {
              "mdi_plus_score": self.feature_importances_by_tree_
          }
      else:
          feature_importances_dict = self.feature_importances_by_tree_
      stability_dict = {}
      for (
          scoring_fn_name,
          feature_importances_by_tree,
      ) in feature_importances_dict.items():
          n_trees = feature_importances_by_tree.shape[1]
          fi_scores_boot_ls = []
          for b in range(B):
              bootstrap_sample = np.random.choice(
                  n_trees, n_trees, replace=True
              )
              fi_scores_boot_ls.append(
                  feature_importances_by_tree[bootstrap_sample].mean(
                      axis=1
                  )
              )
          fi_scores_boot = pd.concat(fi_scores_boot_ls, axis=1)
          stability_results = {"scorer": [scoring_fn_name]}
          for metric_name, metric_fun in metrics.items():
              stability_results[metric_name] = [
                  np.mean(pdist(fi_scores_boot.T, metric=metric_fun))
              ]
          stability_dict[scoring_fn_name] = pd.DataFrame(
              stability_results
          )
      stability_df = pd.concat(stability_dict, axis=0).reset_index(
          drop=True
      )
      if single_scoring_fn:
          stability_df = stability_df.drop(columns=["scorer"])
      return stability_df
  
imports: import numpy
  import pandas
  from scipy.spatial.distance import pdist
  from functools import partial
  from .ppms import PartialPredictionModelBase, GenericRegressorPPM, GenericClassifierPPM
  from .block_transformers import _blocked_train_test_split
  from .ranking_stability import tauAP_b, rbo
label_desc: Use image node_id 3 for calling the ForestMDIPlus obj's underlying member method code with example usage: obj.get_stability_scores(B, metrics) and returns: stability_df