class_docstr: ``torch.optim.Optimizer`` wrapper that adds additional functionality to clip per
  sample gradients and add Gaussian noise.
  
  Can be used with any ``torch.optim.Optimizer`` subclass as an underlying optimizer.
  ``DPOptimzer`` assumes that parameters over which it performs optimization belong
  to GradSampleModule and therefore have the ``grad_sample`` attribute.
  
  On a high level ``DPOptimizer``'s step looks like this:
  1) Aggregate ``p.grad_sample`` over all parameters to calculate per sample norms
  2) Clip ``p.grad_sample`` so that per sample norm is not above threshold
  3) Aggregate clipped per sample gradients into ``p.grad``
  4) Add Gaussian noise to ``p.grad`` calibrated to a given noise multiplier and
  max grad norm limit (``std = noise_multiplier * max_grad_norm``).
  5) Call underlying optimizer to perform optimization step
  
  Examples:
      >>> module = MyCustomModel()
      >>> optimizer = torch.optim.SGD(module.parameters(), lr=0.1)
      >>> dp_optimizer = DPOptimizer(
      ...     optimizer=optimizer,
      ...     noise_multiplier=1.0,
      ...     max_grad_norm=1.0,
      ...     expected_batch_size=4,
      ... )
docstr: Applies given ``loss_reduction`` to ``p.grad``.
  
  Does nothing if ``loss_reduction="sum"``. Divides gradients by
  ``self.expected_batch_size`` if ``loss_reduction="mean"``
docstr_tok: ["Applies","given","``","loss_reduction","``","to","``","p.grad","``",".","Does","nothing","if","``","loss_reduction=","''","sum","''","``",".","Divides","gradients","by","``","self.expected_batch_size","``","if","``","loss_reduction=","''","mean","''","``"]
code: def scale_grad(self):
      """
      Applies given ``loss_reduction`` to ``p.grad``.
  
      Does nothing if ``loss_reduction="sum"``. Divides gradients by
      ``self.expected_batch_size`` if ``loss_reduction="mean"``
      """
      if self.loss_reduction == "mean":
          for p in self.params:
              p.grad /= (
                  self.expected_batch_size * self.accumulated_iterations
              )
  
imports: from __future__ import annotations
  import logging
  from typing import Callable, List, Optional, Union
  import torch
  from opacus.optimizers.utils import params
  from opt_einsum.contract import contract
  from torch import nn
  from torch.optim import Optimizer
label_desc: Use image node_id 11 for calling the DPOptimizer obj's underlying member method code with example usage: obj.scale_grad() without return types