class_docstr: Base class for pairs learners.
  
  Attributes
  ----------
  threshold_ : `float`
    If the distance metric between two points is lower than this threshold,
    points will be classified as similar, otherwise they will be
    classified as dissimilar.
class_docstr_tok: ["Base","class","for","pairs","learners",".","Attributes","--","--","--","--","--","threshold_",":","`","float","`","If","the","distance","metric","between","two","points","is","lower","than","this","threshold",",","points","will","be","classified","as","similar",",","otherwise","they","will","be","classified","as","dissimilar","."]
docstr: Decision threshold calibration for pairwise binary classification
  
  Method that calibrates the decision threshold (cutoff point) of the metric
  learner. This threshold will then be used when calling the method
  `predict`. The methods for picking cutoff points make use of traditional
  binary classification evaluation statistics such as the true positive and
  true negative rates and F-scores. The threshold will be found to maximize
  the chosen score on the validation set ``(pairs_valid, y_valid)``.
  
  See more in the :ref:`User Guide <calibration>`.
  
  Parameters
  ----------
  strategy : str, optional (default='accuracy')
    The strategy to use for choosing the cutoff threshold.
  
    'accuracy'
        Selects a decision threshold that maximizes the accuracy.
    'f_beta'
        Selects a decision threshold that maximizes the f_beta score,
        with beta given by the parameter `beta`.
    'max_tpr'
        Selects a decision threshold that yields the highest true positive
        rate with true negative rate at least equal to the value of the
        parameter `min_rate`.
    'max_tnr'
        Selects a decision threshold that yields the highest true negative
        rate with true positive rate at least equal to the value of the
        parameter `min_rate`.
  
  beta : float in [0, 1], optional (default=None)
    Beta value to be used in case strategy == 'f_beta'.
  
  min_rate : float in [0, 1] or None, (default=None)
    In case strategy is 'max_tpr' or 'max_tnr' this parameter must be set
    to specify the minimal value for the true negative rate or true positive
    rate respectively that needs to be achieved.
  
  pairs_valid : array-like, shape=(n_pairs_valid, 2, n_features)
    The validation set of pairs to use to set the threshold.
  
  y_valid : array-like, shape=(n_pairs_valid,)
    The labels of the pairs of the validation set to use to set the
    threshold. They must be +1 for positive pairs and -1 for negative pairs.
  
  References
  ----------
  .. [1] Receiver-operating characteristic (ROC) plots: a fundamental
         evaluation tool in clinical medicine, MH Zweig, G Campbell -
         Clinical chemistry, 1993
  
  .. [2] Most of the code of this function is from scikit-learn's PR #10117
  
  See Also
  --------
  sklearn.calibration : scikit-learn's module for calibrating classifiers
docstr_tok: ["Decision","threshold","calibration","for","pairwise","binary","classification","Method","that","calibrates","the","decision","threshold","(","cutoff","point",")","of","the","metric","learner",".","This","threshold","will","then","be","used","when","calling","the","method","`","predict","`",".","The","methods","for","picking","cutoff","points","make","use","of","traditional","binary","classification","evaluation","statistics","such","as","the","true","positive","and","true","negative","rates","and","F-scores",".","The","threshold","will","be","found","to","maximize","the","chosen","score","on","the","validation","set","``","(","pairs_valid",",","y_valid",")","``",".","See","more","in","the",":","ref",":","`","User","Guide","<","calibration",">","`",".","Parameters","--","--","--","--","--","strategy",":","str",",","optional","(","default='accuracy","'",")","The","strategy","to","use","for","choosing","the","cutoff","threshold",".","'accuracy'","Selects","a","decision","threshold","that","maximizes","the","accuracy",".","'f_beta'","Selects","a","decision","threshold","that","maximizes","the","f_beta","score",",","with","beta","given","by","the","parameter","`","beta","`",".","'max_tpr'","Selects","a","decision","threshold","that","yields","the","highest","true","positive","rate","with","true","negative","rate","at","least","equal","to","the","value","of","the","parameter","`","min_rate","`",".","'max_tnr'","Selects","a","decision","threshold","that","yields","the","highest","true","negative","rate","with","true","positive","rate","at","least","equal","to","the","value","of","the","parameter","`","min_rate","`",".","beta",":","float","in","[","0",",","1","]",",","optional","(","default=None",")","Beta","value","to","be","used","in","case","strategy","==","'f_beta","'",".","min_rate",":","float","in","[","0",",","1","]","or","None",",","(","default=None",")","In","case","strategy","is","'max_tpr","'","or","'max_tnr","'","this","parameter","must","be","set","to","specify","the","minimal","value","for","the","true","negative","rate","or","true","positive","rate","respectively","that","needs","to","be","achieved",".","pairs_valid",":","array-like",",","shape=","(","n_pairs_valid",",","2",",","n_features",")","The","validation","set","of","pairs","to","use","to","set","the","threshold",".","y_valid",":","array-like",",","shape=","(","n_pairs_valid",",",")","The","labels","of","the","pairs","of","the","validation","set","to","use","to","set","the","threshold",".","They","must","be","+1","for","positive","pairs","and","-1","for","negative","pairs",".","References","--","--","--","--","--","..","[","1","]","Receiver-operating","characteristic","(","ROC",")","plots",":","a","fundamental","evaluation","tool","in","clinical","medicine",",","MH","Zweig",",","G","Campbell","-","Clinical","chemistry",",","1993","..","[","2","]","Most","of","the","code","of","this","function","is","from","scikit-learn","'s","PR","#","10117","See","Also","--","--","--","--","sklearn.calibration",":","scikit-learn","'s","module","for","calibrating","classifiers"]
code: def calibrate_threshold(
      self,
      pairs_valid,
      y_valid,
      strategy="accuracy",
      min_rate=None,
      beta=1.0,
  ):
      """Decision threshold calibration for pairwise binary classification
  
      Method that calibrates the decision threshold (cutoff point) of the metric
      learner. This threshold will then be used when calling the method
      `predict`. The methods for picking cutoff points make use of traditional
      binary classification evaluation statistics such as the true positive and
      true negative rates and F-scores. The threshold will be found to maximize
      the chosen score on the validation set ``(pairs_valid, y_valid)``.
  
      See more in the :ref:`User Guide <calibration>`.
  
      Parameters
      ----------
      strategy : str, optional (default='accuracy')
        The strategy to use for choosing the cutoff threshold.
  
        'accuracy'
            Selects a decision threshold that maximizes the accuracy.
        'f_beta'
            Selects a decision threshold that maximizes the f_beta score,
            with beta given by the parameter `beta`.
        'max_tpr'
            Selects a decision threshold that yields the highest true positive
            rate with true negative rate at least equal to the value of the
            parameter `min_rate`.
        'max_tnr'
            Selects a decision threshold that yields the highest true negative
            rate with true positive rate at least equal to the value of the
            parameter `min_rate`.
  
      beta : float in [0, 1], optional (default=None)
        Beta value to be used in case strategy == 'f_beta'.
  
      min_rate : float in [0, 1] or None, (default=None)
        In case strategy is 'max_tpr' or 'max_tnr' this parameter must be set
        to specify the minimal value for the true negative rate or true positive
        rate respectively that needs to be achieved.
  
      pairs_valid : array-like, shape=(n_pairs_valid, 2, n_features)
        The validation set of pairs to use to set the threshold.
  
      y_valid : array-like, shape=(n_pairs_valid,)
        The labels of the pairs of the validation set to use to set the
        threshold. They must be +1 for positive pairs and -1 for negative pairs.
  
      References
      ----------
      .. [1] Receiver-operating characteristic (ROC) plots: a fundamental
             evaluation tool in clinical medicine, MH Zweig, G Campbell -
             Clinical chemistry, 1993
  
      .. [2] Most of the code of this function is from scikit-learn's PR #10117
  
      See Also
      --------
      sklearn.calibration : scikit-learn's module for calibrating classifiers
      """
      check_is_fitted(self, "preprocessor_")
  
      self._validate_calibration_params(strategy, min_rate, beta)
  
      pairs_valid, y_valid = self._prepare_inputs(
          pairs_valid, y_valid, type_of_inputs="tuples"
      )
  
      n_samples = pairs_valid.shape[0]
      if strategy == "accuracy":
          scores = self.decision_function(pairs_valid)
          scores_sorted_idces = np.argsort(scores)[::-1]
          scores_sorted = scores[scores_sorted_idces]
          # true labels ordered by decision_function value: (higher first)
          y_ordered = y_valid[scores_sorted_idces]
          # we need to add a threshold that will reject all points
          scores_sorted = np.concatenate(
              [[scores_sorted[0] + 1], scores_sorted]
          )
  
          # finds the threshold that maximizes the accuracy:
          cum_tp = stable_cumsum(
              y_ordered == 1
          )  # cumulative number of true
          # positives
          # we need to add the point where all samples are rejected:
          cum_tp = np.concatenate([[0.0], cum_tp])
          cum_tn_inverted = stable_cumsum(y_ordered[::-1] == -1)
          cum_tn = np.concatenate([[0.0], cum_tn_inverted])[::-1]
          cum_accuracy = (cum_tp + cum_tn) / n_samples
          imax = np.argmax(cum_accuracy)
          # we set the threshold to the lowest accepted score
          # note: we are working with negative distances but we want the threshold
          # to be with respect to the actual distances so we take minus sign
          self.threshold_ = -scores_sorted[imax]
          # note: if the best is to reject all points it's already one of the
          # thresholds (scores_sorted[0])
          return self
  
      if strategy == "f_beta":
          precision, recall, thresholds = precision_recall_curve(
              y_valid, self.decision_function(pairs_valid), pos_label=1
          )
  
          # here the thresholds are decreasing
          # We ignore the warnings here, in the same taste as
          # https://github.com/scikit-learn/scikit-learn/blob/62d205980446a1abc1065
          # f4332fd74eee57fcf73/sklearn/metrics/classification.py#L1284
          with np.errstate(divide="ignore", invalid="ignore"):
              f_beta = (
                  (1 + beta**2)
                  * (precision * recall)
                  / (beta**2 * precision + recall)
              )
          # We need to set nans to zero otherwise they will be considered higher
          # than the others (also discussed in https://github.com/scikit-learn/
          # scikit-learn/pull/10117/files#r262115773)
          f_beta[np.isnan(f_beta)] = 0.0
          imax = np.argmax(f_beta)
          # we set the threshold to the lowest accepted score
          # note: we are working with negative distances but we want the threshold
          # to be with respect to the actual distances so we take minus sign
          self.threshold_ = -thresholds[imax]
          # Note: we don't need to deal with rejecting all points (i.e. threshold =
          # max_scores + 1), since this can never happen to be optimal
          # (see a more detailed discussion in test_calibrate_threshold_extreme)
          return self
  
      fpr, tpr, thresholds = roc_curve(
          y_valid, self.decision_function(pairs_valid), pos_label=1
      )
      # here the thresholds are decreasing
      fpr, tpr, thresholds = fpr, tpr, thresholds
  
      if strategy in ["max_tpr", "max_tnr"]:
          if strategy == "max_tpr":
              indices = np.where(1 - fpr >= min_rate)[0]
              imax = np.argmax(tpr[indices])
  
          if strategy == "max_tnr":
              indices = np.where(tpr >= min_rate)[0]
              imax = np.argmax(1 - fpr[indices])
  
          imax_valid = indices[imax]
          # note: we are working with negative distances but we want the threshold
          # to be with respect to the actual distances so we take minus sign
          if indices[imax] == len(
              thresholds
          ):  # we want to accept everything
              self.threshold_ = -(thresholds[imax_valid] - 1)
          else:
              # thanks to roc_curve, the first point will always be max_scores
              # + 1, see: https://github.com/scikit-learn/scikit-learn/pull/13523
              self.threshold_ = -thresholds[imax_valid]
          return self
  
imports: from sklearn.base import BaseEstimator, ClassifierMixin
  from sklearn.utils.extmath import stable_cumsum
  from sklearn.utils.validation import _is_arraylike, check_is_fitted
  from sklearn.metrics import roc_auc_score, roc_curve, precision_recall_curve
  import numpy
  from abc import ABCMeta, abstractmethod
  from ._util import ArrayIndexer, check_input, validate_vector
  import warnings
label_desc: Use image node_id 5 for calling the _PairsClassifierMixin obj's underlying member method code with example usage: obj.calibrate_threshold(pairs_valid, y_valid, strategy, min_rate, beta) and returns: self, self, self