class_docstr: Quantitative Testing with Concept Activation Vectors (TCAV).
  
  TCAV is an interpretability method which allows users to define a concept
  by selecting a set of representative examples and trains a classifier at
  different model layers to determine whether that concept has any influence
  over the predictions of the model (https://arxiv.org/pdf/1711.11279.pdf).
  If so, it can be measured as an aggregate score, and also individual points
  can be measured by how much of that concept they contain, at a given layer.
  
  The original implementation can be found at:
  https://github.com/tensorflow/tcav
  
  This component requires that the following fields in the model spec. Field
  names like `layer` are placeholders; you can call them whatever you like,
  and as with other LIT components multiple segments are supported.
    Output:
      - Embeddings (`emb_layer`) to return the input embeddings
          for a layer
      - Gradients (`grad_layer`) to return the gradients w.r.t.
          `emb_layer`
      - Gradients class (`grad_class`) to return the label that gradients
        were computed for. This is usually a CategoryLabel, but can be anything
        since it will just be fed back into the model.
      - MulticlassPreds (`probas`)
  
  **Note: TCAV calls the model multiple times. It is _strongly_ recommended that
  you use a caching model (i.e., wrap your model in a `CachingModelWrapper`) to
  optimize performance.**
docstr: Runs the TCAV method given the params in the inputs and config.
  
  Args:
    inputs: all examples in the dataset.
    model: the model being explained.
    dataset: the dataset which the current examples belong to.
    model_outputs: optional model outputs from calling model.predict(inputs).
    config: a config which should specify: {
        'concept_set_ids': [list of ids to use in concept set]
        'class_to_explain': [gradient class to explain],
        'grad_layer': [the Gradient field key of the layer to explain],
        'random_state': [an optional seed to make outputs deterministic]
        'test_size': [Percentage of the example set to use in the LM test set]
        'negative_set_ids': [optional list of ids to use as negative set] }
  
  Returns:
    A JsonDict containing the TCAV scores, directional derivatives,
    statistical test p-values, and LM accuracies.
  
  Raises:
    ValueError: configured `grad_layer` is not actually of type `Gradients`
docstr_tok: ["Runs","the","TCAV","method","given","the","params","in","the","inputs","and","config",".","Args",":","inputs",":","all","examples","in","the","dataset",".","model",":","the","model","being","explained",".","dataset",":","the","dataset","which","the","current","examples","belong","to",".","model_outputs",":","optional","model","outputs","from","calling","model.predict","(","inputs",")",".","config",":","a","config","which","should","specify",":","{","'concept_set_ids","'",":","[","list","of","ids","to","use","in","concept","set","]","'class_to_explain","'",":","[","gradient","class","to","explain","]",",","'grad_layer","'",":","[","the","Gradient","field","key","of","the","layer","to","explain","]",",","'random_state","'",":","[","an","optional","seed","to","make","outputs","deterministic","]","'test_size","'",":","[","Percentage","of","the","example","set","to","use","in","the","LM","test","set","]","'negative_set_ids","'",":","[","optional","list","of","ids","to","use","as","negative","set","]","}","Returns",":","A","JsonDict","containing","the","TCAV","scores",",","directional","derivatives",",","statistical","test","p-values",",","and","LM","accuracies",".","Raises",":","ValueError",":","configured","`","grad_layer","`","is","not","actually","of","type","`","Gradients","`"]
code: def run(
      self,
      inputs: Sequence[JsonDict],
      model: lit_model.Model,
      dataset: lit_dataset.Dataset,
      model_outputs: Optional[list[JsonDict]] = None,
      config: Optional[JsonDict] = None,
  ) -> Optional[list[JsonDict]]:
      """Runs the TCAV method given the params in the inputs and config.
  
      Args:
        inputs: all examples in the dataset.
        model: the model being explained.
        dataset: the dataset which the current examples belong to.
        model_outputs: optional model outputs from calling model.predict(inputs).
        config: a config which should specify: {
            'concept_set_ids': [list of ids to use in concept set]
            'class_to_explain': [gradient class to explain],
            'grad_layer': [the Gradient field key of the layer to explain],
            'random_state': [an optional seed to make outputs deterministic]
            'test_size': [Percentage of the example set to use in the LM test set]
            'negative_set_ids': [optional list of ids to use as negative set] }
  
      Returns:
        A JsonDict containing the TCAV scores, directional derivatives,
        statistical test p-values, and LM accuracies.
  
      Raises:
        ValueError: configured `grad_layer` is not actually of type `Gradients`
      """
      if not config:
          raise TypeError("config must be provided")
      tcav_config = TCAVConfig(
          **{
              k: v
              for k, v in config.items()
              if k in _TCAV_CONFIG_FIELDS
          }
      )
  
      # TODO(b/171513556): get these from the Dataset object once indices are
      # available there.
      dataset_examples = inputs
  
      # Get this layer's output spec keys for gradients and embeddings.
      grad_layer = tcav_config.grad_layer
      output_spec = model.output_spec()
      field_spec = output_spec.get(grad_layer)
  
      if not isinstance(field_spec, types.Gradients):
          raise ValueError(
              f"Configured grad_layer, {grad_layer}, must be a "
              "Gradients field"
          )
  
      field_spec = cast(types.Gradients, field_spec)
      emb_layer = field_spec.grad_for
      # Get the class that the gradients were computed for.
      grad_class_key = field_spec.grad_target_field_key
  
      predictions = self.get_predictions(
          model, dataset_examples, model_outputs
      )
  
      # If CAV is provided in config, then only calculate CAV similarity for
      # provided datapoints.
      if tcav_config.cav is not None:
          return [
              {
                  "cos_sim": self._get_cos_sim(
                      np.array(tcav_config.cav), predictions, emb_layer
                  )
              }
          ]
  
      ids_set = set(tcav_config.concept_set_ids)
      concept_set_preds = [
          preds
          for ex, preds in zip(dataset_examples, predictions)
          if ex["_id"] in ids_set
      ]
  
      if tcav_config.negative_set_ids:
          negative_ids_set = set(tcav_config.negative_set_ids)
          negative_set_preds = [
              preds
              for ex, preds in zip(dataset_examples, predictions)
              if ex["_id"] in negative_ids_set
          ]
          return self._run_relative_tcav(
              grad_layer,
              emb_layer,
              grad_class_key,
              concept_set_preds,
              negative_set_preds,
              predictions,
              tcav_config,
          )
      else:
          non_concept_set_preds = [
              preds
              for ex, preds in zip(dataset_examples, predictions)
              if ex["_id"] not in ids_set
          ]
          return self._run_default_tcav(
              grad_layer,
              emb_layer,
              grad_class_key,
              concept_set_preds,
              non_concept_set_preds,
              predictions,
              tcav_config,
          )
  
imports: from collections.abc import Sequence
  import dataclasses
  import math
  import random
  from typing import Any, Optional, cast
  from lit_nlp.api import components
  from lit_nlp.api import dataset
  from lit_nlp.api import model
  from lit_nlp.api import types
  from lit_nlp.lib import utils
  import numpy
  import scipy.stats
  import sklearn.linear_model
  import sklearn.model_selection
label_desc: Use image node_id 4 for calling the TCAV obj's underlying member method code with example usage: obj.run(inputs, model, dataset, model_outputs, config) and returns: list, self, self