class_docstr: This class implements ConceptInterpreter abstract class using an
  approach called Testing with Concept Activation Vectors (TCAVs),
  as described in the paper:
  https://arxiv.org/abs/1711.11279
  
  TCAV scores for a given layer, a list of concepts and input example
  are computed using the dot product between prediction's layer
  sensitivities for given input examples and Concept Activation Vectors
  (CAVs) in that same layer.
  
  CAVs are defined as vectors that are orthogonal to the classification boundary
  hyperplane that separate given concepts in a given layer from each other.
  For a given layer, CAVs are computed by training a classifier that uses the
  layer activation vectors for a set of concept examples as input examples and
  concept ids as corresponding input labels. Trained weights of
  that classifier represent CAVs.
  
  CAVs are represented as a learned weight matrix with the dimensionality
  C X F, where:
  F represents the number of input features in the classifier.
  C is the number of concepts used for the classification. Concept
  ids are used as labels for concept examples during the training.
  
  We can use any layer attribution algorithm to compute layer sensitivities
  of a model prediction.
  For example, the gradients of an output prediction w.r.t. the outputs of
  the layer.
  The CAVs and the Sensitivities (SENS) are used to compute the TCAV score:
  
  0. TCAV = CAV • SENS, a dot product between those two vectors
  
  The final TCAV score can be computed by aggregating the TCAV scores
  for each input concept based on the sign or magnitude of the tcav scores.
  
  1. sign_count_score = | TCAV > 0 | / | TCAV |
  2. magnitude_score = SUM(ABS(TCAV * (TCAV > 0))) / SUM(ABS(TCAV))
class_docstr_tok: ["This","class","implements","ConceptInterpreter","abstract","class","using","an","approach","called","Testing","with","Concept","Activation","Vectors","(","TCAVs",")",",","as","described","in","the","paper",":","https",":","\/\/arxiv.org\/abs\/1711.11279","TCAV","scores","for","a","given","layer",",","a","list","of","concepts","and","input","example","are","computed","using","the","dot","product","between","prediction","'s","layer","sensitivities","for","given","input","examples","and","Concept","Activation","Vectors","(","CAVs",")","in","that","same","layer",".","CAVs","are","defined","as","vectors","that","are","orthogonal","to","the","classification","boundary","hyperplane","that","separate","given","concepts","in","a","given","layer","from","each","other",".","For","a","given","layer",",","CAVs","are","computed","by","training","a","classifier","that","uses","the","layer","activation","vectors","for","a","set","of","concept","examples","as","input","examples","and","concept","ids","as","corresponding","input","labels",".","Trained","weights","of","that","classifier","represent","CAVs",".","CAVs","are","represented","as","a","learned","weight","matrix","with","the","dimensionality","C","X","F",",","where",":","F","represents","the","number","of","input","features","in","the","classifier",".","C","is","the","number","of","concepts","used","for","the","classification",".","Concept","ids","are","used","as","labels","for","concept","examples","during","the","training",".","We","can","use","any","layer","attribution","algorithm","to","compute","layer","sensitivities","of","a","model","prediction",".","For","example",",","the","gradients","of","an","output","prediction","w.r.t",".","the","outputs","of","the","layer",".","The","CAVs","and","the","Sensitivities","(","SENS",")","are","used","to","compute","the","TCAV","score",":","0",".","TCAV","=","CAV","\u2022","SENS",",","a","dot","product","between","those","two","vectors","The","final","TCAV","score","can","be","computed","by","aggregating","the","TCAV","scores","for","each","input","concept","based","on","the","sign","or","magnitude","of","the","tcav","scores",".","1.","sign_count_score","=","|","TCAV",">","0","|","\/","|","TCAV","|","2.","magnitude_score","=","SUM","(","ABS","(","TCAV","*","(","TCAV",">","0",")",")",")","\/","SUM","(","ABS","(","TCAV",")",")"]
docstr: This method computes magnitude and sign-based TCAV scores for each
  experimental sets in `experimental_sets` list.
  TCAV scores are computed using a dot product between layer attribution
  scores for specific predictions and CAV vectors.
  
  Args:
  
      inputs (Tensor or tuple[Tensor, ...]): Inputs for which predictions
              are performed and attributions are computed.
              If model takes a single tensor as
              input, a single input tensor should be provided.
              If model takes multiple tensors as
              input, a tuple of the input tensors should be provided.
              It is assumed that for all given input tensors,
              dimension 0 corresponds to the number of examples
              (aka batch size), and if multiple input tensors are
              provided, the examples must be aligned appropriately.
      experimental_sets (list[list[Concept]]): A list of list of Concept
              instances.
      target (int, tuple, Tensor, or list, optional): Output indices for
              which attributions are computed (for classification cases,
              this is usually the target class).
              If the network returns a scalar value per example,
              no target index is necessary.
              For general 2D outputs, targets can be either:
  
              - a single integer or a tensor containing a single
                  integer, which is applied to all input examples
              - a list of integers or a 1D tensor, with length matching
                  the number of examples in inputs (dim 0). Each integer
                  is applied as the target for the corresponding example.
  
              For outputs with > 2 dimensions, targets can be either:
  
              - A single tuple, which contains #output_dims - 1
                  elements. This target index is applied to all examples.
              - A list of tuples with length equal to the number of
                  examples in inputs (dim 0), and each tuple containing
                  #output_dims - 1 elements. Each tuple is applied as the
                  target for the corresponding example.
  
      additional_forward_args (Any, optional): Extra arguments that are passed to
               model when computing the attributions for `inputs`
               w.r.t. layer output.
               Default: None
      processes (int, optional): The number of processes to be created. if
              processes is larger than one then CAV computations will be
              performed in parallel using the number of processes equal to
              `processes`. Otherwise, CAV computations will be performed
              sequential.
              Default:None
      **kwargs (Any, optional): A list of arguments that are passed to layer
              attribution algorithm's attribute method. This could be for
              example `n_steps` in case of integrated gradients.
              Default: None
  
  Returns:
      results (dict): A dictionary of sign and magnitude -based tcav scores
              for each concept set per layer.
              The order of TCAV scores in the resulting tensor for each
              experimental set follows the order in which concepts
              are passed in `experimental_sets` input argument.
  
  results example::
      >>> #
      >>> # scores =
      >>> # {'0-1':
      >>> #     {'inception4c':
      >>> #         {'sign_count': tensor([0.5800, 0.4200]),
      >>> #          'magnitude': tensor([0.6613, 0.3387])},
      >>> #      'inception4d':
      >>> #         {'sign_count': tensor([0.6200, 0.3800]),
      >>> #           'magnitude': tensor([0.7707, 0.2293])}}),
      >>> #  '0-2':
      >>> #     {'inception4c':
      >>> #         {'sign_count': tensor([0.6200, 0.3800]),
      >>> #          'magnitude': tensor([0.6806, 0.3194])},
      >>> #      'inception4d':
      >>> #         {'sign_count': tensor([0.6400, 0.3600]),
      >>> #          'magnitude': tensor([0.6563, 0.3437])}})})
      >>> #
docstr_tok: ["This","method","computes","magnitude","and","sign-based","TCAV","scores","for","each","experimental","sets","in","`","experimental_sets","`","list",".","TCAV","scores","are","computed","using","a","dot","product","between","layer","attribution","scores","for","specific","predictions","and","CAV","vectors",".","Args",":","inputs","(","Tensor","or","tuple","[","Tensor",",","...","]",")",":","Inputs","for","which","predictions","are","performed","and","attributions","are","computed",".","If","model","takes","a","single","tensor","as","input",",","a","single","input","tensor","should","be","provided",".","If","model","takes","multiple","tensors","as","input",",","a","tuple","of","the","input","tensors","should","be","provided",".","It","is","assumed","that","for","all","given","input","tensors",",","dimension","0","corresponds","to","the","number","of","examples","(","aka","batch","size",")",",","and","if","multiple","input","tensors","are","provided",",","the","examples","must","be","aligned","appropriately",".","experimental_sets","(","list","[","list","[","Concept","]","]",")",":","A","list","of","list","of","Concept","instances",".","target","(","int",",","tuple",",","Tensor",",","or","list",",","optional",")",":","Output","indices","for","which","attributions","are","computed","(","for","classification","cases",",","this","is","usually","the","target","class",")",".","If","the","network","returns","a","scalar","value","per","example",",","no","target","index","is","necessary",".","For","general","2D","outputs",",","targets","can","be","either",":","-","a","single","integer","or","a","tensor","containing","a","single","integer",",","which","is","applied","to","all","input","examples","-","a","list","of","integers","or","a","1D","tensor",",","with","length","matching","the","number","of","examples","in","inputs","(","dim","0",")",".","Each","integer","is","applied","as","the","target","for","the","corresponding","example",".","For","outputs","with",">","2","dimensions",",","targets","can","be","either",":","-","A","single","tuple",",","which","contains","#","output_dims","-","1","elements",".","This","target","index","is","applied","to","all","examples",".","-","A","list","of","tuples","with","length","equal","to","the","number","of","examples","in","inputs","(","dim","0",")",",","and","each","tuple","containing","#","output_dims","-","1","elements",".","Each","tuple","is","applied","as","the","target","for","the","corresponding","example",".","additional_forward_args","(","Any",",","optional",")",":","Extra","arguments","that","are","passed","to","model","when","computing","the","attributions","for","`","inputs","`","w.r.t",".","layer","output",".","Default",":","None","processes","(","int",",","optional",")",":","The","number","of","processes","to","be","created",".","if","processes","is","larger","than","one","then","CAV","computations","will","be","performed","in","parallel","using","the","number","of","processes","equal","to","`","processes","`",".","Otherwise",",","CAV","computations","will","be","performed","sequential",".","Default",":","None","*","*","kwargs","(","Any",",","optional",")",":","A","list","of","arguments","that","are","passed","to","layer","attribution","algorithm","'s","attribute","method",".","This","could","be","for","example","`","n_steps","`","in","case","of","integrated","gradients",".","Default",":","None","Returns",":","results","(","dict",")",":","A","dictionary","of","sign","and","magnitude","-based","tcav","scores","for","each","concept","set","per","layer",".","The","order","of","TCAV","scores","in","the","resulting","tensor","for","each","experimental","set","follows","the","order","in","which","concepts","are","passed","in","`","experimental_sets","`","input","argument",".","results","example",":",":",">",">",">","#",">",">",">","#","scores","=",">",">",">","#","{","'","0-1","'",":",">",">",">","#","{","'inception4c","'",":",">",">",">","#","{","'sign_count","'",":","tensor","(","[","0.5800",",","0.4200","]",")",",",">",">",">","#","'magnitude","'",":","tensor","(","[","0.6613",",","0.3387","]",")","}",",",">",">",">","#","'inception4d","'",":",">",">",">","#","{","'sign_count","'",":","tensor","(","[","0.6200",",","0.3800","]",")",",",">",">",">","#","'magnitude","'",":","tensor","(","[","0.7707",",","0.2293","]",")","}","}",")",",",">",">",">","#","'","0-2","'",":",">",">",">","#","{","'inception4c","'",":",">",">",">","#","{","'sign_count","'",":","tensor","(","[","0.6200",",","0.3800","]",")",",",">",">",">","#","'magnitude","'",":","tensor","(","[","0.6806",",","0.3194","]",")","}",",",">",">",">","#","'inception4d","'",":",">",">",">","#","{","'sign_count","'",":","tensor","(","[","0.6400",",","0.3600","]",")",",",">",">",">","#","'magnitude","'",":","tensor","(","[","0.6563",",","0.3437","]",")","}","}",")","}",")",">",">",">","#"]
code: def interpret(
      self,
      inputs: TensorOrTupleOfTensorsGeneric,
      experimental_sets: List[List[Concept]],
      target: TargetType = None,
      additional_forward_args: Any = None,
      processes: int = None,
      **kwargs: Any,
  ) -> Dict[str, Dict[str, Dict[str, Tensor]]]:
      r"""
      This method computes magnitude and sign-based TCAV scores for each
      experimental sets in `experimental_sets` list.
      TCAV scores are computed using a dot product between layer attribution
      scores for specific predictions and CAV vectors.
  
      Args:
  
          inputs (Tensor or tuple[Tensor, ...]): Inputs for which predictions
                  are performed and attributions are computed.
                  If model takes a single tensor as
                  input, a single input tensor should be provided.
                  If model takes multiple tensors as
                  input, a tuple of the input tensors should be provided.
                  It is assumed that for all given input tensors,
                  dimension 0 corresponds to the number of examples
                  (aka batch size), and if multiple input tensors are
                  provided, the examples must be aligned appropriately.
          experimental_sets (list[list[Concept]]): A list of list of Concept
                  instances.
          target (int, tuple, Tensor, or list, optional): Output indices for
                  which attributions are computed (for classification cases,
                  this is usually the target class).
                  If the network returns a scalar value per example,
                  no target index is necessary.
                  For general 2D outputs, targets can be either:
  
                  - a single integer or a tensor containing a single
                      integer, which is applied to all input examples
                  - a list of integers or a 1D tensor, with length matching
                      the number of examples in inputs (dim 0). Each integer
                      is applied as the target for the corresponding example.
  
                  For outputs with > 2 dimensions, targets can be either:
  
                  - A single tuple, which contains #output_dims - 1
                      elements. This target index is applied to all examples.
                  - A list of tuples with length equal to the number of
                      examples in inputs (dim 0), and each tuple containing
                      #output_dims - 1 elements. Each tuple is applied as the
                      target for the corresponding example.
  
          additional_forward_args (Any, optional): Extra arguments that are passed to
                   model when computing the attributions for `inputs`
                   w.r.t. layer output.
                   Default: None
          processes (int, optional): The number of processes to be created. if
                  processes is larger than one then CAV computations will be
                  performed in parallel using the number of processes equal to
                  `processes`. Otherwise, CAV computations will be performed
                  sequential.
                  Default:None
          **kwargs (Any, optional): A list of arguments that are passed to layer
                  attribution algorithm's attribute method. This could be for
                  example `n_steps` in case of integrated gradients.
                  Default: None
  
      Returns:
          results (dict): A dictionary of sign and magnitude -based tcav scores
                  for each concept set per layer.
                  The order of TCAV scores in the resulting tensor for each
                  experimental set follows the order in which concepts
                  are passed in `experimental_sets` input argument.
  
      results example::
          >>> #
          >>> # scores =
          >>> # {'0-1':
          >>> #     {'inception4c':
          >>> #         {'sign_count': tensor([0.5800, 0.4200]),
          >>> #          'magnitude': tensor([0.6613, 0.3387])},
          >>> #      'inception4d':
          >>> #         {'sign_count': tensor([0.6200, 0.3800]),
          >>> #           'magnitude': tensor([0.7707, 0.2293])}}),
          >>> #  '0-2':
          >>> #     {'inception4c':
          >>> #         {'sign_count': tensor([0.6200, 0.3800]),
          >>> #          'magnitude': tensor([0.6806, 0.3194])},
          >>> #      'inception4d':
          >>> #         {'sign_count': tensor([0.6400, 0.3600]),
          >>> #          'magnitude': tensor([0.6563, 0.3437])}})})
          >>> #
  
      """
      assert "attribute_to_layer_input" not in kwargs, (
          "Please, set `attribute_to_layer_input` flag as a constructor "
          "argument to TCAV class. In that case it will be applied "
          "consistently to both layer activation and layer attribution methods."
      )
      self.compute_cavs(experimental_sets, processes=processes)
  
      scores: Dict[str, Dict[str, Dict[str, Tensor]]] = defaultdict(
          lambda: defaultdict()
      )
  
      # Retrieves the lengths of the experimental sets so that we can sort
      # them by the length and compute TCAV scores in batches.
      exp_set_lens = np.array(
          list(map(lambda exp_set: len(exp_set), experimental_sets)),
          dtype=object,
      )
      exp_set_lens_arg_sort = np.argsort(exp_set_lens)
  
      # compute offsets using sorted lengths using their indices
      exp_set_lens_sort = exp_set_lens[exp_set_lens_arg_sort]
      exp_set_offsets_bool = [False] + list(
          exp_set_lens_sort[:-1] == exp_set_lens_sort[1:]
      )
      exp_set_offsets = []
      for i, offset in enumerate(exp_set_offsets_bool):
          if not offset:
              exp_set_offsets.append(i)
  
      exp_set_offsets.append(len(exp_set_lens))
  
      # sort experimental sets using the length of the concepts in each set
      experimental_sets_sorted = np.array(
          experimental_sets, dtype=object
      )[exp_set_lens_arg_sort]
  
      for layer in self.layers:
          layer_module = _get_module_from_name(self.model, layer)
          self.layer_attr_method.layer = layer_module
          attribs = self.layer_attr_method.attribute.__wrapped__(  # type: ignore
              self.layer_attr_method,  # self
              inputs,
              target=target,
              additional_forward_args=additional_forward_args,
              attribute_to_layer_input=self.attribute_to_layer_input,
              **kwargs,
          )
  
          attribs = _format_tensor_into_tuples(attribs)
          # n_inputs x n_features
          attribs = torch.cat(
              [
                  torch.reshape(attrib, (attrib.shape[0], -1))
                  for attrib in attribs
              ],
              dim=1,
          )
  
          # n_experiments x n_concepts x n_features
          cavs = []
          classes = []
          for concepts in experimental_sets:
              concepts_key = concepts_to_str(concepts)
              cavs_stats = cast(
                  Dict[str, Any], self.cavs[concepts_key][layer].stats
              )
              cavs.append(
                  cavs_stats["weights"].float().detach().tolist()
              )
              classes.append(cavs_stats["classes"])
  
          # sort cavs and classes using the length of the concepts in each set
          cavs_sorted = np.array(cavs, dtype=object)[
              exp_set_lens_arg_sort
          ]
          classes_sorted = np.array(classes, dtype=object)[
              exp_set_lens_arg_sort
          ]
          i = 0
          while i < len(exp_set_offsets) - 1:
              cav_subset = np.array(
                  cavs_sorted[
                      exp_set_offsets[i] : exp_set_offsets[i + 1]
                  ],
                  dtype=object,
              ).tolist()
              classes_subset = classes_sorted[
                  exp_set_offsets[i] : exp_set_offsets[i + 1]
              ].tolist()
  
              # n_experiments x n_concepts x n_features
              cav_subset = torch.tensor(cav_subset)
              cav_subset = cav_subset.to(attribs.device)
              assert len(cav_subset.shape) == 3, (
                  "cav should have 3 dimensions: n_experiments x "
                  "n_concepts x n_features."
              )
  
              experimental_subset_sorted = experimental_sets_sorted[
                  exp_set_offsets[i] : exp_set_offsets[i + 1]
              ]
              self._tcav_sub_computation(
                  scores,
                  layer,
                  attribs,
                  cav_subset,
                  classes_subset,
                  experimental_subset_sorted,
              )
              i += 1
  
      return scores
  
imports: from collections import defaultdict
  from typing import Any, cast, Dict, List, Set, Tuple, Union
  import numpy
  import torch
  import torch.multiprocessing
  from captum._utils.av import AV
  from captum._utils.common import _format_tensor_into_tuples, _get_module_from_name
  from captum._utils.typing import TargetType, TensorOrTupleOfTensorsGeneric
  from captum.attr import LayerActivation, LayerAttribution, LayerGradientXActivation
  from captum.concept._core.cav import CAV
  from captum.concept._core.concept import Concept, ConceptInterpreter
  from captum.concept._utils.classifier import Classifier, DefaultClassifier
  from captum.concept._utils.common import concepts_to_str
  from captum.log import log_usage
  from torch import Tensor
  from torch.nn import Module
  from torch.utils.data import DataLoader, Dataset
label_desc: Use image node_id 7 for calling the TCAV obj's underlying member method code with example usage: obj.interpret(inputs, experimental_sets, target, additional_forward_args, processes) and returns: scores