class_docstr: This is a computationally-inefficient implementation that computes the type of
  "infinitesimal" influence scores defined in the paper "Understanding Black-box
  Predictions via Influence Functions" by Koh et al
  (https://arxiv.org/pdf/1703.04730.pdf). The computational bottleneck in computing
  infinitesimal influence scores is computing inverse Hessian-vector products, as can
  be seen from its definition in `InfluenceFunctionBase`. This implementation is
  inefficient / naive in that it explicitly forms the Hessian :math`H`, unlike other
  implementations which compute inverse Hessian-vector products without explicitly
  forming the Hessian. The purpose of this implementation is to have a way to
  generate the "ground-truth" influence scores, to which other implementations,
  which are more efficient but return only approximations of the influence score, can
  be compared.
  
  This implementation computes a low-rank approximation of the inverse Hessian, i.e.
  a tall and skinny (with width k) matrix :math`R` such that
  :math`H^{-1} \approx RR'`, where k is small. In particular, let :math`L` be the
  matrix of width k whose columns contain the top-k eigenvectors of :math`H`, and let
  :math`V` be the k by k matrix whose diagonals contain the corresponding eigenvalues.
  This implementation lets :math`R=LV^{-1}L'`. Thus, the core computational step is
  computing the top-k eigenvalues / eigenvectors.
  
  This low-rank approximation is useful for several reasons:
  - It avoids numerical issues associated with inverting small eigenvalues.
  - Since the influence score is given by
    :math`\nabla_\theta L(x)' H^{-1} \nabla_\theta L(z)`, which is approximated by
    :math`(\nabla_\theta L(x)' R) (\nabla_\theta L(z)' R)`, we can compute an
    "influence embedding" for a given example :math`x`, :math`\nabla_\theta L(x)' R`,
    such that the influence score of one example on another is approximately the
    dot-product of their respective embeddings.
  
  This implementation is "naive" in that it computes the top-k eigenvalues /
  eigenvectors by explicitly forming the Hessian, converting it to a 2D tensor,
  computing its eigenvectors / eigenvalues, and then sorting. See documentation of the
  `_retrieve_projections_naive_influence_function` method for more details.
docstr: Computes "embedding" vectors for all examples in a single batch, or a
  `Dataloader` that yields batches. These embedding vectors are constructed so
  that the influence score of a training example on a test example is simply the
  dot-product of their corresponding vectors. In both cases, a batch should be
  small enough so that a backwards pass for a batch does not lead to
  out-of-memory errors.
  
  In more detail, the embedding vector for an example `x` is
  :math`\nabla_\theta L(x)' R`, where :math`R` is as defined in this class'
  description. The embeddings for a batch of examples are computed by assembling
  :math`\nabla_\theta L(x)` for all examples `x` in the batch as rows in a 2D
  tensor, and right-multiplying by `R`.
  
  If `aggregate` is True, the *sum* of the vectors for all examples is returned,
  instead of the vectors for each example. This can be useful for computing the
  influence of a given training example on the total loss over a validation
  dataset, because due to properties of the dot-product, this influence is the
  dot-product of the training example's vector with the sum of the vectors in the
  validation dataset. Also, by doing the sum aggregation within this method as
  opposed to outside of it (by computing all vectors for the validation dataset,
  then taking the sum) allows memory usage to be reduced.
  
  Args:
      inputs_dataset (Tuple, or DataLoader): Either a single tuple of any, or a
              `DataLoader`, where each batch yielded is a tuple of any. In
              either case, the tuple represents a single batch, where the last
              element is assumed to be the labels for the batch. That is,
              `model(*batch[0:-1])` produces the output for `model`, and
              and `batch[-1]` are the labels, if any. Here, `model` is model
              provided in initialization. This is the same assumption made for
              each batch yielded by training dataset `train_dataset`.
      aggregate (bool): Whether to return the sum of the vectors for all
              examples, as opposed to vectors for each example.
      show_progress (bool, optional): Computation of vectors can take a long
              time if `inputs_dataset` represents many examples. If
              `show_progress`is true, the progress of this computation will be
              displayed. In particular, the number of batches for which
              vectors have been computed will be displayed. It will try to
              use tqdm if available for advanced features (e.g. time estimation).
              Otherwise, it will fallback to a simple output of progress.
              Default: False
      return_on_cpu (bool, optional): Whether to return the vectors on the cpu.
              If None or False, is set to the device that the model is on.
              Default: None
      test (bool, optional): Whether to compute the vectors using the loss
              function `test_loss_fn` provided in initialization (instead of
              `loss_fn`). This argument does not matter if `test_loss_fn` was
              not provided, as in this case, `test_loss_fn` and `loss_fn` are the
              same.
  
  Returns:
      intermediate_quantities (Tensor): This is a 2D tensor with shape
              `(N, projection_dim)`, where `N` is the total number of examples in
              `inputs_dataset`, and `projection_dim` was provided in
              initialization. Each row contains the vector for a different
              example.
docstr_tok: ["Computes","``","embedding","''","vectors","for","all","examples","in","a","single","batch",",","or","a","`","Dataloader","`","that","yields","batches",".","These","embedding","vectors","are","constructed","so","that","the","influence","score","of","a","training","example","on","a","test","example","is","simply","the","dot-product","of","their","corresponding","vectors",".","In","both","cases",",","a","batch","should","be","small","enough","so","that","a","backwards","pass","for","a","batch","does","not","lead","to","out-of-memory","errors",".","In","more","detail",",","the","embedding","vector","for","an","example","`","x","`","is",":","math","`","\\nabla_\\theta","L","(","x",")","'","R","`",",","where",":","math","`","R","`","is","as","defined","in","this","class'","description",".","The","embeddings","for","a","batch","of","examples","are","computed","by","assembling",":","math","`","\\nabla_\\theta","L","(","x",")","`","for","all","examples","`","x","`","in","the","batch","as","rows","in","a","2D","tensor",",","and","right-multiplying","by","`","R","`",".","If","`","aggregate","`","is","True",",","the","*","sum","*","of","the","vectors","for","all","examples","is","returned",",","instead","of","the","vectors","for","each","example",".","This","can","be","useful","for","computing","the","influence","of","a","given","training","example","on","the","total","loss","over","a","validation","dataset",",","because","due","to","properties","of","the","dot-product",",","this","influence","is","the","dot-product","of","the","training","example","'s","vector","with","the","sum","of","the","vectors","in","the","validation","dataset",".","Also",",","by","doing","the","sum","aggregation","within","this","method","as","opposed","to","outside","of","it","(","by","computing","all","vectors","for","the","validation","dataset",",","then","taking","the","sum",")","allows","memory","usage","to","be","reduced",".","Args",":","inputs_dataset","(","Tuple",",","or","DataLoader",")",":","Either","a","single","tuple","of","any",",","or","a","`","DataLoader","`",",","where","each","batch","yielded","is","a","tuple","of","any",".","In","either","case",",","the","tuple","represents","a","single","batch",",","where","the","last","element","is","assumed","to","be","the","labels","for","the","batch",".","That","is",",","`","model","(","*","batch","[","0",":","-1","]",")","`","produces","the","output","for","`","model","`",",","and","and","`","batch","[","-1","]","`","are","the","labels",",","if","any",".","Here",",","`","model","`","is","model","provided","in","initialization",".","This","is","the","same","assumption","made","for","each","batch","yielded","by","training","dataset","`","train_dataset","`",".","aggregate","(","bool",")",":","Whether","to","return","the","sum","of","the","vectors","for","all","examples",",","as","opposed","to","vectors","for","each","example",".","show_progress","(","bool",",","optional",")",":","Computation","of","vectors","can","take","a","long","time","if","`","inputs_dataset","`","represents","many","examples",".","If","`","show_progress","`","is","true",",","the","progress","of","this","computation","will","be","displayed",".","In","particular",",","the","number","of","batches","for","which","vectors","have","been","computed","will","be","displayed",".","It","will","try","to","use","tqdm","if","available","for","advanced","features","(","e.g",".","time","estimation",")",".","Otherwise",",","it","will","fallback","to","a","simple","output","of","progress",".","Default",":","False","return_on_cpu","(","bool",",","optional",")",":","Whether","to","return","the","vectors","on","the","cpu",".","If","None","or","False",",","is","set","to","the","device","that","the","model","is","on",".","Default",":","None","test","(","bool",",","optional",")",":","Whether","to","compute","the","vectors","using","the","loss","function","`","test_loss_fn","`","provided","in","initialization","(","instead","of","`","loss_fn","`",")",".","This","argument","does","not","matter","if","`","test_loss_fn","`","was","not","provided",",","as","in","this","case",",","`","test_loss_fn","`","and","`","loss_fn","`","are","the","same",".","Returns",":","intermediate_quantities","(","Tensor",")",":","This","is","a","2D","tensor","with","shape","`","(","N",",","projection_dim",")","`",",","where","`","N","`","is","the","total","number","of","examples","in","`","inputs_dataset","`",",","and","`","projection_dim","`","was","provided","in","initialization",".","Each","row","contains","the","vector","for","a","different","example","."]
code: def compute_intermediate_quantities(
      self,
      inputs_dataset: Union[Tuple[Any, ...], DataLoader],
      aggregate: bool = False,
      show_progress: bool = False,
      return_on_cpu: bool = True,
      test: bool = False,
  ) -> Tensor:
      r"""
      Computes "embedding" vectors for all examples in a single batch, or a
      `Dataloader` that yields batches. These embedding vectors are constructed so
      that the influence score of a training example on a test example is simply the
      dot-product of their corresponding vectors. In both cases, a batch should be
      small enough so that a backwards pass for a batch does not lead to
      out-of-memory errors.
  
      In more detail, the embedding vector for an example `x` is
      :math`\nabla_\theta L(x)' R`, where :math`R` is as defined in this class'
      description. The embeddings for a batch of examples are computed by assembling
      :math`\nabla_\theta L(x)` for all examples `x` in the batch as rows in a 2D
      tensor, and right-multiplying by `R`.
  
      If `aggregate` is True, the *sum* of the vectors for all examples is returned,
      instead of the vectors for each example. This can be useful for computing the
      influence of a given training example on the total loss over a validation
      dataset, because due to properties of the dot-product, this influence is the
      dot-product of the training example's vector with the sum of the vectors in the
      validation dataset. Also, by doing the sum aggregation within this method as
      opposed to outside of it (by computing all vectors for the validation dataset,
      then taking the sum) allows memory usage to be reduced.
  
      Args:
          inputs_dataset (Tuple, or DataLoader): Either a single tuple of any, or a
                  `DataLoader`, where each batch yielded is a tuple of any. In
                  either case, the tuple represents a single batch, where the last
                  element is assumed to be the labels for the batch. That is,
                  `model(*batch[0:-1])` produces the output for `model`, and
                  and `batch[-1]` are the labels, if any. Here, `model` is model
                  provided in initialization. This is the same assumption made for
                  each batch yielded by training dataset `train_dataset`.
          aggregate (bool): Whether to return the sum of the vectors for all
                  examples, as opposed to vectors for each example.
          show_progress (bool, optional): Computation of vectors can take a long
                  time if `inputs_dataset` represents many examples. If
                  `show_progress`is true, the progress of this computation will be
                  displayed. In particular, the number of batches for which
                  vectors have been computed will be displayed. It will try to
                  use tqdm if available for advanced features (e.g. time estimation).
                  Otherwise, it will fallback to a simple output of progress.
                  Default: False
          return_on_cpu (bool, optional): Whether to return the vectors on the cpu.
                  If None or False, is set to the device that the model is on.
                  Default: None
          test (bool, optional): Whether to compute the vectors using the loss
                  function `test_loss_fn` provided in initialization (instead of
                  `loss_fn`). This argument does not matter if `test_loss_fn` was
                  not provided, as in this case, `test_loss_fn` and `loss_fn` are the
                  same.
  
      Returns:
          intermediate_quantities (Tensor): This is a 2D tensor with shape
                  `(N, projection_dim)`, where `N` is the total number of examples in
                  `inputs_dataset`, and `projection_dim` was provided in
                  initialization. Each row contains the vector for a different
                  example.
      """
      # if `inputs_dataset` is not a `DataLoader`, turn it into one.
      inputs_dataset = _format_inputs_dataset(inputs_dataset)
  
      if show_progress:
          inputs_dataset = _progress_bar_constructor(
              self,
              inputs_dataset,
              "inputs_dataset",
              "intermediate quantities",
          )
      # infer model / data device through model
      if return_on_cpu is None or (not return_on_cpu):
          return_device = self.model_device
      else:
          return_device = torch.device("cpu")
  
      # as described in the description for `NaiveInfluenceFunction`, the embedding
      # for an example `x` is :math`\nabla_\theta L(x)' R`.
      # `_basic_computation_naive_influence_function` returns a 2D tensor where
      # each row is :math`\nabla_\theta L(x)'` for a different example `x` in a
      # batch. therefore, we right-multiply its output with `R` to get the embeddings
      # for a batch, and then concatenate the per-batch embeddings to get embeddings
      # for the entire dataset.
  
      # choose the correct loss function and reduction type based on `test`
      loss_fn = self.test_loss_fn if test else self.loss_fn
      reduction_type = (
          self.test_reduction_type if test else self.reduction_type
      )
  
      # define a helper function that returns the embeddings for a batch
      def get_batch_embeddings(batch):
          # if `self.R` is on cpu, and `self.model_device` was not cpu, this implies
          # `self.R` was too large to fit in gpu memory, and we should do the matrix
          # multiplication of the batch jacobians with `self.R` separately for each
          # column of `self.R`, to avoid moving the entire `self.R` to gpu all at
          # once and running out of gpu memory
          batch_jacobians = _basic_computation_naive_influence_function(
              self, batch[0:-1], batch[-1], loss_fn, reduction_type
          )
          if self.R.device == torch.device(
              "cpu"
          ) and self.model_device != torch.device("cpu"):
              return torch.stack(
                  [
                      torch.matmul(
                          batch_jacobians,
                          R_col.to(batch_jacobians.device),
                      )
                      for R_col in self.R.T
                  ],
                  dim=1,
              ).to(return_device)
          else:
              return torch.matmul(batch_jacobians, self.R).to(
                  device=return_device
              )
  
      # using `get_batch_embeddings` and a helper, return all the vectors or their
      # sum, depending on `aggregate`
      return _get_dataset_embeddings_intermediate_quantities_influence_function(
          get_batch_embeddings,
          inputs_dataset,
          aggregate,
      )
  
imports: import functools
  from abc import abstractmethod
  from operator import add
  from typing import Any, Callable, List, Optional, Tuple, Union
  import torch
  import torch.nn
  from captum._utils.gradient import _extract_parameters_from_layers
  from captum.influence._core.influence import DataInfluence
  from captum.influence._utils.common import _check_loss_fn, _compute_batch_loss_influence_function_base, _compute_jacobian_sample_wise_grads_per_batch, _dataset_fn, _flatten_params, _format_inputs_dataset, _functional_call, _get_k_most_influential_helper, _influence_batch_intermediate_quantities_influence_function, _influence_helper_intermediate_quantities_influence_function, _influence_route_to_helpers, _load_flexible_state_dict, _params_to_names, _progress_bar_constructor, _self_influence_helper_intermediate_quantities_influence_function, _set_active_parameters, _top_eigen, _unflatten_params_factory, KMostInfluentialResults
  from captum.log import log_usage
  from torch import Tensor
  from torch.nn import Module
  from torch.utils.data import DataLoader, Dataset
  from tqdm import tqdm
label_desc: Use image node_id 3 for calling the NaiveInfluenceFunction obj's underlying member method code with example usage: obj.compute_intermediate_quantities(inputs_dataset, aggregate, show_progress, return_on_cpu, test) and returns: _get_dataset_embeddings_intermediate_quantities_influence_function, torch, torch