class_docstr: The standard BN layer computes stats across all examples in a GPU. In some
  cases it is desirable to compute stats across only a subset of examples
  (e.g., in multigrid training https://arxiv.org/abs/1912.00998).
  SubBatchNorm3d splits the batch dimension into N splits, and run BN on
  each of them separately (so that the stats are computed on each subset of
  examples (1/N of batch) independently. During evaluation, it aggregates
  the stats from all splits into one BN.
class_docstr_tok: ["The","standard","BN","layer","computes","stats","across","all","examples","in","a","GPU",".","In","some","cases","it","is","desirable","to","compute","stats","across","only","a","subset","of","examples","(","e.g.",",","in","multigrid","training","https",":","\/\/arxiv.org\/abs\/1912.00998",")",".","SubBatchNorm3d","splits","the","batch","dimension","into","N","splits",",","and","run","BN","on","each","of","them","separately","(","so","that","the","stats","are","computed","on","each","subset","of","examples","(","1\/N","of","batch",")","independently",".","During","evaluation",",","it","aggregates","the","stats","from","all","splits","into","one","BN","."]
docstr: Args:
      num_splits (int): number of splits.
      args (list): other arguments.
docstr_tok: ["Args",":","num_splits","(","int",")",":","number","of","splits",".","args","(","list",")",":","other","arguments","."]
code: def __init__(self, num_splits, **args):
      """
      Args:
          num_splits (int): number of splits.
          args (list): other arguments.
      """
      super(SubBatchNorm3d, self).__init__()
      self.num_splits = num_splits
      num_features = args["num_features"]
      # Keep only one set of weight and bias.
      if args.get("affine", True):
          self.affine = True
          args["affine"] = False
          self.weight = torch.nn.Parameter(torch.ones(num_features))
          self.bias = torch.nn.Parameter(torch.zeros(num_features))
      else:
          self.affine = False
      self.bn = nn.BatchNorm3d(**args)
      args["num_features"] = num_features * num_splits
      self.split_bn = nn.BatchNorm3d(**args)
  
file_path: SlowFast/slowfast/models/batchnorm_helper.py
imports: from functools import partial
  import torch
  import torch.nn
  from pytorchvideo.layers.batch_norm import NaiveSyncBatchNorm1d, NaiveSyncBatchNorm3d
label_desc: Use image node_id 1 to create a new SubBatchNorm3d object from inherited base classes: nn with example: obj = SubBatchNorm3d(num_splits)