docstr: Computes self influence scores for the examples in `inputs`, which is
  either a single batch or a Pytorch `DataLoader` that yields batches. Therefore,
  the computed self influence scores are *not* for the examples in training
  dataset `train_dataset` (unlike when computing self influence scores using the
  `influence` method). Note that if `inputs` is a single batch, this
  will call `model` on that single batch, and if `inputs` yields
  batches, this will call `model` on each batch that is yielded. Therefore,
  please ensure that for both cases, the batch(es) that `model` is called
  with are not too large, so that there will not be an out-of-memory error. This
  implementation performs an outer iteration over checkpoints, and an inner
  iteration over all batches that `inputs` represents. The pros of this
  implementation are that the checkpoints do not need to be loaded too many
  times.
  
  Args:
      batches (tuple or DataLoader): Either a single tuple of any, or a
              `DataLoader`, where each batch yielded is a tuple of any. In
              either case, the tuple represents a single batch, where the last
              element is assumed to be the labels for the batch. That is,
              `model(*batch[0:-1])` produces the output for `model`,
              and `batch[-1]` are the labels, if any. This is the same
              assumption made for each batch yielded by training dataset
              `train_dataset`. Please see documentation for the
              `train_dataset` argument to `TracInCP.__init__` for
              more details on the assumed structure of a batch.
      show_progress (bool, optional): Computation of self influence scores can
              take a long time if `inputs` represents many examples. If
              `show_progress` is true, the progress of this computation will be
              displayed. In more detail, this computation will iterate over all
              checkpoints (provided as the `checkpoints` initialization argument)
              in an outer loop, and iterate over all batches that
              `inputs` represents in an inner loop. Thus if
              `show_progress` is True, the progress of both the outer
              iteration and the inner iterations will be displayed. To show
              progress, it will try to use tqdm if available for advanced
              features (e.g. time estimation). Otherwise, it will fallback to a
              simple output of progress.
              Default: False
  
  Returns:
      self_influence_scores (Tensor): This is a 1D tensor containing the self
              influence scores of all examples in `inputs`, regardless of
              whether it represents a single batch or a `DataLoader` that yields
              batches.
docstr_tok: ["Computes","self","influence","scores","for","the","examples","in","`","inputs","`",",","which","is","either","a","single","batch","or","a","Pytorch","`","DataLoader","`","that","yields","batches",".","Therefore",",","the","computed","self","influence","scores","are","*","not","*","for","the","examples","in","training","dataset","`","train_dataset","`","(","unlike","when","computing","self","influence","scores","using","the","`","influence","`","method",")",".","Note","that","if","`","inputs","`","is","a","single","batch",",","this","will","call","`","model","`","on","that","single","batch",",","and","if","`","inputs","`","yields","batches",",","this","will","call","`","model","`","on","each","batch","that","is","yielded",".","Therefore",",","please","ensure","that","for","both","cases",",","the","batch","(","es",")","that","`","model","`","is","called","with","are","not","too","large",",","so","that","there","will","not","be","an","out-of-memory","error",".","This","implementation","performs","an","outer","iteration","over","checkpoints",",","and","an","inner","iteration","over","all","batches","that","`","inputs","`","represents",".","The","pros","of","this","implementation","are","that","the","checkpoints","do","not","need","to","be","loaded","too","many","times",".","Args",":","batches","(","tuple","or","DataLoader",")",":","Either","a","single","tuple","of","any",",","or","a","`","DataLoader","`",",","where","each","batch","yielded","is","a","tuple","of","any",".","In","either","case",",","the","tuple","represents","a","single","batch",",","where","the","last","element","is","assumed","to","be","the","labels","for","the","batch",".","That","is",",","`","model","(","*","batch","[","0",":","-1","]",")","`","produces","the","output","for","`","model","`",",","and","`","batch","[","-1","]","`","are","the","labels",",","if","any",".","This","is","the","same","assumption","made","for","each","batch","yielded","by","training","dataset","`","train_dataset","`",".","Please","see","documentation","for","the","`","train_dataset","`","argument","to","`","TracInCP.__init__","`","for","more","details","on","the","assumed","structure","of","a","batch",".","show_progress","(","bool",",","optional",")",":","Computation","of","self","influence","scores","can","take","a","long","time","if","`","inputs","`","represents","many","examples",".","If","`","show_progress","`","is","true",",","the","progress","of","this","computation","will","be","displayed",".","In","more","detail",",","this","computation","will","iterate","over","all","checkpoints","(","provided","as","the","`","checkpoints","`","initialization","argument",")","in","an","outer","loop",",","and","iterate","over","all","batches","that","`","inputs","`","represents","in","an","inner","loop",".","Thus","if","`","show_progress","`","is","True",",","the","progress","of","both","the","outer","iteration","and","the","inner","iterations","will","be","displayed",".","To","show","progress",",","it","will","try","to","use","tqdm","if","available","for","advanced","features","(","e.g",".","time","estimation",")",".","Otherwise",",","it","will","fallback","to","a","simple","output","of","progress",".","Default",":","False","Returns",":","self_influence_scores","(","Tensor",")",":","This","is","a","1D","tensor","containing","the","self","influence","scores","of","all","examples","in","`","inputs","`",",","regardless","of","whether","it","represents","a","single","batch","or","a","`","DataLoader","`","that","yields","batches","."]
code: def _self_influence_by_checkpoints(
      self,
      inputs: Union[Tuple[Any, ...], DataLoader],
      show_progress: bool = False,
  ) -> Tensor:
      """
      Computes self influence scores for the examples in `inputs`, which is
      either a single batch or a Pytorch `DataLoader` that yields batches. Therefore,
      the computed self influence scores are *not* for the examples in training
      dataset `train_dataset` (unlike when computing self influence scores using the
      `influence` method). Note that if `inputs` is a single batch, this
      will call `model` on that single batch, and if `inputs` yields
      batches, this will call `model` on each batch that is yielded. Therefore,
      please ensure that for both cases, the batch(es) that `model` is called
      with are not too large, so that there will not be an out-of-memory error. This
      implementation performs an outer iteration over checkpoints, and an inner
      iteration over all batches that `inputs` represents. The pros of this
      implementation are that the checkpoints do not need to be loaded too many
      times.
  
      Args:
          batches (tuple or DataLoader): Either a single tuple of any, or a
                  `DataLoader`, where each batch yielded is a tuple of any. In
                  either case, the tuple represents a single batch, where the last
                  element is assumed to be the labels for the batch. That is,
                  `model(*batch[0:-1])` produces the output for `model`,
                  and `batch[-1]` are the labels, if any. This is the same
                  assumption made for each batch yielded by training dataset
                  `train_dataset`. Please see documentation for the
                  `train_dataset` argument to `TracInCP.__init__` for
                  more details on the assumed structure of a batch.
          show_progress (bool, optional): Computation of self influence scores can
                  take a long time if `inputs` represents many examples. If
                  `show_progress` is true, the progress of this computation will be
                  displayed. In more detail, this computation will iterate over all
                  checkpoints (provided as the `checkpoints` initialization argument)
                  in an outer loop, and iterate over all batches that
                  `inputs` represents in an inner loop. Thus if
                  `show_progress` is True, the progress of both the outer
                  iteration and the inner iterations will be displayed. To show
                  progress, it will try to use tqdm if available for advanced
                  features (e.g. time estimation). Otherwise, it will fallback to a
                  simple output of progress.
                  Default: False
  
      Returns:
          self_influence_scores (Tensor): This is a 1D tensor containing the self
                  influence scores of all examples in `inputs`, regardless of
                  whether it represents a single batch or a `DataLoader` that yields
                  batches.
      """
      # If `inputs` is not a `DataLoader`, turn it into one.
      inputs = _format_inputs_dataset(inputs)
  
      # If `show_progress` is true, create an outer progress bar that keeps track of
      # how many checkpoints have been processed
      if show_progress:
          # Try to determine length of inner progress bar if possible, with a default
          # of `None`.
          inputs_len = None
          try:
              inputs_len = len(inputs)
          except TypeError:
              warnings.warn(
                  "Unable to determine the number of batches in `inputs`. "
                  "Therefore, if showing the progress of the computation of self "
                  "influence scores, only the number of batches processed can be "
                  "displayed, and not the percentage completion of the computation, "
                  "nor any time estimates."
              )
  
      def calculate_via_vector_norm(layer_jacobian):
          # Helper to efficiently calculate vector norm if pytorch version permits.
          return (
              torch.linalg.vector_norm(
                  layer_jacobian,
                  dim=list(range(1, len(layer_jacobian.shape))),
              )
              ** 2
          )
  
      def calculate_via_flatten(layer_jacobian):
          return torch.sum(
              layer_jacobian.flatten(start_dim=1) ** 2, dim=1
          )
  
      def get_checkpoint_contribution(checkpoint):
          # This function returns a 1D tensor representing the contribution to the
          # self influence score for the given checkpoint, for all batches in
          # `inputs`. The length of the 1D tensor is the total number of
          # examples in `inputs`.
          assert (
              checkpoint is not None
          ), "None returned from `checkpoints`, cannot load."
  
          learning_rate = self.checkpoints_load_func(
              self.model, checkpoint
          )
  
          # This will store a list of the contribution of the self influence score
          # from each batch. Each element is a 1D tensor of length batch_size - the
          # batch size of each batch in `inputs` (they do not need to be all
          # the same)
          checkpoint_contribution = []
  
          _inputs = inputs
          # If `show_progress` is true, create an inner progress bar that keeps track
          # of how many batches have been processed for the current checkpoint
          if show_progress:
              _inputs = progress(
                  inputs,
                  desc=(
                      f"Using {self.get_name()} to compute self "
                      "influence. Processing batch"
                  ),
                  total=inputs_len,
              )
  
          for batch in _inputs:
              layer_jacobians = self._basic_computation_tracincp(
                  batch[0:-1],
                  batch[-1],
                  self.loss_fn,
                  self.reduction_type,
              )
  
              # Note that all variables in this function are for an entire batch.
              # Each `layer_jacobian` in `layer_jacobians` corresponds to a different
              # layer. `layer_jacobian` is the jacobian w.r.t to a given layer's
              # parameters. If the given layer's parameters are of shape *, then
              # `layer_jacobian` is of shape (batch_size, *). For each layer, we need
              # the squared jacobian for each example. So we square the jacobian and
              # sum over all dimensions except the 0-th (the batch dimension). We then
              # sum the contribution over all layers.  For Pytorch > 1.10 we use the
              # optimized torch.linalg.vector_norm as opposed to the explicit flatten.
  
              calculate_fn = calculate_via_flatten
              if _parse_version(torch.__version__) >= (1, 10, 0):
                  calculate_fn = calculate_via_vector_norm
  
              checkpoint_contribution.append(
                  torch.sum(
                      torch.stack(
                          [
                              calculate_fn(layer_jacobian)
                              for layer_jacobian in layer_jacobians
                          ],
                          dim=0,
                      ),
                      dim=0,
                  )
                  * learning_rate
              )
  
          # We concatenate the contributions from each batch into a single 1D tensor,
          # which represents the contributions for all batches in `inputs`
  
          return torch.cat(checkpoint_contribution, dim=0)
  
      if show_progress:
          checkpoints_progress = progress(
              desc=(
                  f"Using {self.get_name()} to compute self "
                  "influence. Processing checkpoint"
              ),
              total=len(self.checkpoints),
              mininterval=0.0,
          )
      else:
          checkpoints_progress = NullProgress()
      with checkpoints_progress:
          batches_self_tracin_scores = get_checkpoint_contribution(
              self.checkpoints[0]
          )
          checkpoints_progress.update()
          # The self influence score for all examples is the sum of contributions from
          # each checkpoint
          for checkpoint in self.checkpoints[1:]:
              batches_self_tracin_scores += get_checkpoint_contribution(
                  checkpoint
              )
              checkpoints_progress.update()
  
      return batches_self_tracin_scores
  
imports: import glob
  import warnings
  from abc import abstractmethod
  from os.path import join
  from typing import Any, Callable, Iterator, List, Optional, Tuple, Type, Union
  import torch
  from captum._utils.av import AV
  from captum._utils.common import _parse_version
  from captum._utils.progress import NullProgress, progress
  from captum.influence._core.influence import DataInfluence
  from captum.influence._utils.common import _check_loss_fn, _compute_jacobian_sample_wise_grads_per_batch, _format_inputs_dataset, _get_k_most_influential_helper, _gradient_dot_product, _influence_route_to_helpers, _load_flexible_state_dict, _self_influence_by_batches_helper, _set_active_parameters, KMostInfluentialResults
  from captum.log import log_usage
  from torch import Tensor
  from torch.nn import Module
  from torch.utils.data import DataLoader, Dataset
label_desc: Use image node_id 10 for calling the TracInCP obj's underlying member method code with example usage: obj._self_influence_by_checkpoints(inputs, show_progress) and returns: batches_self_tracin_scores, unknown, torch, torch