class_docstr: Implementation of the Model Parameter Randomisation Test (MPRT) by Adebayo et al., 2018.
  
  The MPRT measures the distance between the original attribution and a newly computed
  attribution throughout the process of cascadingly/ independently randomizing the model
  parameters of one layer at a time.
  
  Assumptions:
      - In the original paper multiple distance measures are taken: Spearman rank correlation (with and without abs),
      HOG and SSIM. We have set Spearman as the default value.
  
  References:
      1) Julius Adebayo et al.: "Sanity Checks for Saliency Maps." NeurIPS (2018): 9525-9536.
  
  Attributes:
      -  _name: The name of the metric.
      - _data_applicability: The data types that the metric implementation currently supports.
      - _models: The model types that this metric can work with.
      - score_direction: How to interpret the scores, whether higher/ lower values are considered better.
      - evaluation_category: What property/ explanation quality that this metric measures.
docstr: This implementation represents the main logic of the metric and makes the class object callable.
  It completes instance-wise evaluation of explanations (a_batch) with respect to input data (x_batch),
  output labels (y_batch) and a torch or tensorflow model (model).
  
  Calls general_preprocess() with all relevant arguments, calls
  () on each instance, and saves results to evaluation_scores.
  Calls custom_postprocess() afterwards. Finally returns evaluation_scores.
  
  The content of evaluation_scores will be appended to all_evaluation_scores (list) at the end of
  the evaluation call.
  
  Parameters
  ----------
  model: torch.nn.Module, tf.keras.Model
      A torch or tensorflow model that is subject to explanation.
  x_batch: np.ndarray
      A np.ndarray which contains the input data that are explained.
  y_batch: np.ndarray
      A np.ndarray which contains the output labels that are explained.
  a_batch: np.ndarray, optional
      A np.ndarray which contains pre-computed attributions i.e., explanations.
  s_batch: np.ndarray, optional
      A np.ndarray which contains segmentation masks that matches the input.
  channel_first: boolean, optional
      Indicates of the image dimensions are channel first, or channel last.
      Inferred from the input shape if None.
  explain_func: callable
      Callable generating attributions.
  explain_func_kwargs: dict, optional
      Keyword arguments to be passed to explain_func on call.
  model_predict_kwargs: dict, optional
      Keyword arguments to be passed to the model's predict method.
  softmax: boolean
      Indicates whether to use softmax probabilities or logits in model prediction.
      This is used for this __call__ only and won't be saved as attribute. If None, self.softmax is used.
  device: string
      Indicated the device on which a torch.Tensor is or will be allocated: "cpu" or "gpu".
  kwargs: optional
      Keyword arguments.
  
  Returns
  -------
  evaluation_scores: list
      a list of Any with the evaluation scores of the concerned batch.
  
  Examples:
  --------
      # Minimal imports.
      >> import quantus
      >> from quantus import LeNet
      >> import torch
  
      # Enable GPU.
      >> device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
  
      # Load a pre-trained LeNet classification model (architecture at quantus/helpers/models).
      >> model = LeNet()
      >> model.load_state_dict(torch.load("tutorials/assets/pytests/mnist_model"))
  
      # Load MNIST datasets and make loaders.
      >> test_set = torchvision.datasets.MNIST(root='./sample_data', download=True)
      >> test_loader = torch.utils.data.DataLoader(test_set, batch_size=24)
  
      # Load a batch of inputs and outputs to use for XAI evaluation.
      >> x_batch, y_batch = iter(test_loader).next()
      >> x_batch, y_batch = x_batch.cpu().numpy(), y_batch.cpu().numpy()
  
      # Generate Saliency attributions of the test set batch of the test set.
      >> a_batch_saliency = Saliency(model).attribute(inputs=x_batch, target=y_batch, abs=True).sum(axis=1)
      >> a_batch_saliency = a_batch_saliency.cpu().numpy()
  
      # Initialise the metric and evaluate explanations by calling the metric instance.
      >> metric = Metric(abs=True, normalise=False)
      >> scores = metric(model=model, x_batch=x_batch, y_batch=y_batch, a_batch=a_batch_saliency)
docstr_tok: ["This","implementation","represents","the","main","logic","of","the","metric","and","makes","the","class","object","callable",".","It","completes","instance-wise","evaluation","of","explanations","(","a_batch",")","with","respect","to","input","data","(","x_batch",")",",","output","labels","(","y_batch",")","and","a","torch","or","tensorflow","model","(","model",")",".","Calls","general_preprocess","(",")","with","all","relevant","arguments",",","calls","(",")","on","each","instance",",","and","saves","results","to","evaluation_scores",".","Calls","custom_postprocess","(",")","afterwards",".","Finally","returns","evaluation_scores",".","The","content","of","evaluation_scores","will","be","appended","to","all_evaluation_scores","(","list",")","at","the","end","of","the","evaluation","call",".","Parameters","--","--","--","--","--","model",":","torch.nn.Module",",","tf.keras.Model","A","torch","or","tensorflow","model","that","is","subject","to","explanation",".","x_batch",":","np.ndarray","A","np.ndarray","which","contains","the","input","data","that","are","explained",".","y_batch",":","np.ndarray","A","np.ndarray","which","contains","the","output","labels","that","are","explained",".","a_batch",":","np.ndarray",",","optional","A","np.ndarray","which","contains","pre-computed","attributions","i.e.",",","explanations",".","s_batch",":","np.ndarray",",","optional","A","np.ndarray","which","contains","segmentation","masks","that","matches","the","input",".","channel_first",":","boolean",",","optional","Indicates","of","the","image","dimensions","are","channel","first",",","or","channel","last",".","Inferred","from","the","input","shape","if","None",".","explain_func",":","callable","Callable","generating","attributions",".","explain_func_kwargs",":","dict",",","optional","Keyword","arguments","to","be","passed","to","explain_func","on","call",".","model_predict_kwargs",":","dict",",","optional","Keyword","arguments","to","be","passed","to","the","model","'s","predict","method",".","softmax",":","boolean","Indicates","whether","to","use","softmax","probabilities","or","logits","in","model","prediction",".","This","is","used","for","this","__call__","only","and","wo","n't","be","saved","as","attribute",".","If","None",",","self.softmax","is","used",".","device",":","string","Indicated","the","device","on","which","a","torch.Tensor","is","or","will","be","allocated",":","``","cpu","''","or","``","gpu","''",".","kwargs",":","optional","Keyword","arguments",".","Returns","--","--","--","-","evaluation_scores",":","list","a","list","of","Any","with","the","evaluation","scores","of","the","concerned","batch",".","Examples",":","--","--","--","--","#","Minimal","imports",".",">",">","import","quantus",">",">","from","quantus","import","LeNet",">",">","import","torch","#","Enable","GPU",".",">",">","device","=","torch.device","(","``","cuda:0","''","if","torch.cuda.is_available","(",")","else","``","cpu","''",")","#","Load","a","pre-trained","LeNet","classification","model","(","architecture","at","quantus\/helpers\/models",")",".",">",">","model","=","LeNet","(",")",">",">","model.load_state_dict","(","torch.load","(","``","tutorials\/assets\/pytests\/mnist_model","''",")",")","#","Load","MNIST","datasets","and","make","loaders",".",">",">","test_set","=","torchvision.datasets.MNIST","(","root='.\/sample_data","'",",","download=True",")",">",">","test_loader","=","torch.utils.data.DataLoader","(","test_set",",","batch_size=24",")","#","Load","a","batch","of","inputs","and","outputs","to","use","for","XAI","evaluation",".",">",">","x_batch",",","y_batch","=","iter","(","test_loader",")",".next","(",")",">",">","x_batch",",","y_batch","=","x_batch.cpu","(",")",".numpy","(",")",",","y_batch.cpu","(",")",".numpy","(",")","#","Generate","Saliency","attributions","of","the","test","set","batch","of","the","test","set",".",">",">","a_batch_saliency","=","Saliency","(","model",")",".attribute","(","inputs=x_batch",",","target=y_batch",",","abs=True",")",".sum","(","axis=1",")",">",">","a_batch_saliency","=","a_batch_saliency.cpu","(",")",".numpy","(",")","#","Initialise","the","metric","and","evaluate","explanations","by","calling","the","metric","instance",".",">",">","metric","=","Metric","(","abs=True",",","normalise=False",")",">",">","scores","=","metric","(","model=model",",","x_batch=x_batch",",","y_batch=y_batch",",","a_batch=a_batch_saliency",")"]
code: def __call__(
      self,
      model,
      x_batch: np.ndarray,
      y_batch: np.ndarray,
      a_batch: Optional[np.ndarray] = None,
      s_batch: Optional[np.ndarray] = None,
      channel_first: Optional[bool] = None,
      explain_func: Optional[Callable] = None,
      explain_func_kwargs: Optional[Dict] = None,
      model_predict_kwargs: Optional[Dict] = None,
      softmax: Optional[bool] = False,
      device: Optional[str] = None,
      batch_size: int = 64,
      **kwargs,
  ) -> Union[
      List[float], float, Dict[str, List[float]], Collection[Any]
  ]:
      """
      This implementation represents the main logic of the metric and makes the class object callable.
      It completes instance-wise evaluation of explanations (a_batch) with respect to input data (x_batch),
      output labels (y_batch) and a torch or tensorflow model (model).
  
      Calls general_preprocess() with all relevant arguments, calls
      () on each instance, and saves results to evaluation_scores.
      Calls custom_postprocess() afterwards. Finally returns evaluation_scores.
  
      The content of evaluation_scores will be appended to all_evaluation_scores (list) at the end of
      the evaluation call.
  
      Parameters
      ----------
      model: torch.nn.Module, tf.keras.Model
          A torch or tensorflow model that is subject to explanation.
      x_batch: np.ndarray
          A np.ndarray which contains the input data that are explained.
      y_batch: np.ndarray
          A np.ndarray which contains the output labels that are explained.
      a_batch: np.ndarray, optional
          A np.ndarray which contains pre-computed attributions i.e., explanations.
      s_batch: np.ndarray, optional
          A np.ndarray which contains segmentation masks that matches the input.
      channel_first: boolean, optional
          Indicates of the image dimensions are channel first, or channel last.
          Inferred from the input shape if None.
      explain_func: callable
          Callable generating attributions.
      explain_func_kwargs: dict, optional
          Keyword arguments to be passed to explain_func on call.
      model_predict_kwargs: dict, optional
          Keyword arguments to be passed to the model's predict method.
      softmax: boolean
          Indicates whether to use softmax probabilities or logits in model prediction.
          This is used for this __call__ only and won't be saved as attribute. If None, self.softmax is used.
      device: string
          Indicated the device on which a torch.Tensor is or will be allocated: "cpu" or "gpu".
      kwargs: optional
          Keyword arguments.
  
      Returns
      -------
      evaluation_scores: list
          a list of Any with the evaluation scores of the concerned batch.
  
      Examples:
      --------
          # Minimal imports.
          >> import quantus
          >> from quantus import LeNet
          >> import torch
  
          # Enable GPU.
          >> device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
  
          # Load a pre-trained LeNet classification model (architecture at quantus/helpers/models).
          >> model = LeNet()
          >> model.load_state_dict(torch.load("tutorials/assets/pytests/mnist_model"))
  
          # Load MNIST datasets and make loaders.
          >> test_set = torchvision.datasets.MNIST(root='./sample_data', download=True)
          >> test_loader = torch.utils.data.DataLoader(test_set, batch_size=24)
  
          # Load a batch of inputs and outputs to use for XAI evaluation.
          >> x_batch, y_batch = iter(test_loader).next()
          >> x_batch, y_batch = x_batch.cpu().numpy(), y_batch.cpu().numpy()
  
          # Generate Saliency attributions of the test set batch of the test set.
          >> a_batch_saliency = Saliency(model).attribute(inputs=x_batch, target=y_batch, abs=True).sum(axis=1)
          >> a_batch_saliency = a_batch_saliency.cpu().numpy()
  
          # Initialise the metric and evaluate explanations by calling the metric instance.
          >> metric = Metric(abs=True, normalise=False)
          >> scores = metric(model=model, x_batch=x_batch, y_batch=y_batch, a_batch=a_batch_saliency)
      """
  
      # Run deprecation warnings.
      warn.deprecation_warnings(kwargs)
      warn.check_kwargs(kwargs)
      self.batch_size = batch_size
      data = self.general_preprocess(
          model=model,
          x_batch=x_batch,
          y_batch=y_batch,
          a_batch=a_batch,
          s_batch=s_batch,
          custom_batch=None,
          channel_first=channel_first,
          explain_func=explain_func,
          explain_func_kwargs=explain_func_kwargs,
          model_predict_kwargs=model_predict_kwargs,
          softmax=softmax,
          device=device,
      )
      model: ModelInterface = data["model"]  # type: ignore
      # Here _batch refers to full dataset.
      x_full_dataset = data["x_batch"]
      y_full_dataset = data["y_batch"]
      a_full_dataset = data["a_batch"]
  
      # Results are returned/saved as a dictionary not as a list as in the super-class.
      self.evaluation_scores = {}
  
      # Get number of iterations from number of layers.
      n_layers = model.random_layer_generator_length
      pbar = tqdm(
          total=n_layers * len(x_full_dataset),
          disable=not self.display_progressbar,
      )
      if self.display_progressbar:
          # Set property to False, so we display only 1 pbar.
          self._display_progressbar = False
  
      with pbar as pbar:
          for l_ix, (layer_name, random_layer_model) in enumerate(
              model.get_random_layer_generator(
                  order=self.layer_order, seed=self.seed
              )
          ):
              pbar.desc = layer_name
  
              if l_ix == 0:
                  # Generate explanations on original model in batches.
                  a_original_generator = self.generate_explanations(
                      model.get_model(),
                      x_full_dataset,
                      y_full_dataset,
                      batch_size,
                  )
  
                  # Compute the similarity of explanations of the original model.
                  self.evaluation_scores["original"] = []
                  for a_batch, a_batch_original in zip(
                      self.generate_a_batches(a_full_dataset),
                      a_original_generator,
                  ):
                      for a_instance, a_instance_original in zip(
                          a_batch, a_batch_original
                      ):
                          score = self.evaluate_instance(
                              model=model,
                              x=None,
                              y=None,
                              s=None,
                              a=a_instance,
                              a_perturbed=a_instance_original,
                          )
                          # Save similarity scores in a result dictionary.
                          self.evaluation_scores["original"].append(
                              score
                          )
                          pbar.update(1)
  
              # Skip layers if computing delta.
              if self.skip_layers and (l_ix + 1) < n_layers:
                  continue
  
              self.evaluation_scores[layer_name] = []
  
              # Generate explanations on perturbed model in batches.
              a_perturbed_generator = self.generate_explanations(
                  random_layer_model,
                  x_full_dataset,
                  y_full_dataset,
                  batch_size,
              )
  
              # Compute the similarity of explanations of the perturbed model.
              for a_batch, a_batch_perturbed in zip(
                  self.generate_a_batches(a_full_dataset),
                  a_perturbed_generator,
              ):
                  for a_instance, a_instance_perturbed in zip(
                      a_batch, a_batch_perturbed
                  ):
                      score = self.evaluate_instance(
                          model=random_layer_model,
                          x=None,
                          y=None,
                          s=None,
                          a=a_instance,
                          a_perturbed=a_instance_perturbed,
                      )
                      self.evaluation_scores[layer_name].append(score)
                      pbar.update(1)
  
      if self.return_average_correlation:
          self.evaluation_scores = (
              self.recompute_average_correlation_per_sample()
          )
  
      elif self.return_last_correlation:
          self.evaluation_scores = (
              self.recompute_last_correlation_per_sample()
          )
  
      if self.return_aggregate:
          assert (
              self.return_average_correlation
              or self.return_last_correlation
          ), (
              "Set 'return_average_correlation' or 'return_last_correlation'"
              " to True in order to compute the aggregate evaluation results."
          )
          self.evaluation_scores = [
              self.aggregate_func(self.evaluation_scores)
          ]
  
      # Return all_evaluation_scores according to Quantus.
      self.all_evaluation_scores.append(self.evaluation_scores)
  
      return self.evaluation_scores
  
imports: import sys
  import warnings
  from typing import Any, Callable, Collection, Dict, List, Optional, Union, Tuple, Generator, Iterable
  import numpy
  from tqdm.auto import tqdm
  from sklearn.utils import gen_batches
  from quantus.functions.similarity_func import correlation_spearman
  from quantus.helpers import asserts, warn
  from quantus.helpers.enums import DataType, EvaluationCategory, ModelType, ScoreDirection
  from quantus.helpers.model.model_interface import ModelInterface
  from quantus.metrics.base import Metric
label_desc: Use image node_id 2 for calling the MPRT obj's underlying member method code with example usage: obj.__call__(model, x_batch, y_batch, a_batch, s_batch, channel_first, explain_func, explain_func_kwargs, model_predict_kwargs, softmax, device, batch_size) and returns: self