class_docstr: Helpers for writing expectation functions over tests.
  Each test has a list of testcases, and each testcase has a list of examples.
  Expectation function will act on whole tests, testcases, individual examples, or pairs of examples.
  In any of these, the output of an expectation function for a single example
  is an integer, float, bool, or None, where:
      > 0 (or True) means passed,
      <= 0 or False means fail, and (optionally) the magnitude of the
        failure, indicated by distance from 0, e.g. -10 is worse than -1
      None means the test does not apply, and this should not be counted
docstr: Expectation over pairs of examples, suitable for perturbation tests
  
  Parameters
  ----------
  fn : function
      Arguments: (orig_pred, pred, orig_conf, conf, labels=None, meta=None)
          Orig_pred and orig_conf are the prediction and the confidence
          of the first example in the test case
      Returns: bool, float, or int. See docstring for the Expect class
      for what different values in the output mean.
  Returns
  -------
  function
      Arguments: AbstractTest
      Returns: List of np.arrays
docstr_tok: ["Expectation","over","pairs","of","examples",",","suitable","for","perturbation","tests","Parameters","--","--","--","--","--","fn",":","function","Arguments",":","(","orig_pred",",","pred",",","orig_conf",",","conf",",","labels=None",",","meta=None",")","Orig_pred","and","orig_conf","are","the","prediction","and","the","confidence","of","the","first","example","in","the","test","case","Returns",":","bool",",","float",",","or","int",".","See","docstring","for","the","Expect","class","for","what","different","values","in","the","output","mean",".","Returns","--","--","--","-","function","Arguments",":","AbstractTest","Returns",":","List","of","np.arrays"]
code: def pairwise(fn):
      """Expectation over pairs of examples, suitable for perturbation tests
  
      Parameters
      ----------
      fn : function
          Arguments: (orig_pred, pred, orig_conf, conf, labels=None, meta=None)
              Orig_pred and orig_conf are the prediction and the confidence
              of the first example in the test case
          Returns: bool, float, or int. See docstring for the Expect class
          for what different values in the output mean.
      Returns
      -------
      function
          Arguments: AbstractTest
          Returns: List of np.arrays
      """
  
      def expect_fn(xs, preds, confs, labels=None, meta=None):
          orig_pred = preds[0]
          orig_conf = confs[0]
          return np.array(
              [
                  fn(orig_pred, p, orig_conf, c, l, m)
                  for _, p, c, l, m in iter_with_optional(
                      xs, preds, confs, labels, meta
                  )
              ]
          )
  
      return Expect.testcase(expect_fn)
  
imports: import numpy
  import itertools
label_desc: Use image node_id 4 for calling the Expect obj's underlying member method code with example usage: obj.pairwise(fn) and returns: Expect, np