class_docstr: An implementation of residual addition of an adapter module with its input for the MHA Adapters.
docstr: A basic strategy, comprising of a residual connection over the input, after forward pass by
  the underlying adapter. Additional work is done to pack and unpack the dictionary of inputs and outputs.
  
  Note: The `value` tensor is added to the output of the attention adapter as the residual connection.
  
  Args:
      input: A dictionary of multiple input arguments for the adapter module.
          `query`, `key`, `value`: Original output tensor of the module, or the output of the
           previous adapter (if more than one adapters are enabled).
           `mask`: Attention mask.
           `pos_emb`: Optional positional embedding for relative encoding.
      adapter: The adapter module that is currently required to perform the forward pass.
      module: The calling module, in its entirety. It is a module that implements `AdapterModuleMixin`,
          therefore the strategy can access all other adapters in this module via `module.adapter_layer`.
  
  Returns:
      The result tensor, after one of the active adapters has finished its forward passes.
docstr_tok: ["A","basic","strategy",",","comprising","of","a","residual","connection","over","the","input",",","after","forward","pass","by","the","underlying","adapter",".","Additional","work","is","done","to","pack","and","unpack","the","dictionary","of","inputs","and","outputs",".","Note",":","The","`","value","`","tensor","is","added","to","the","output","of","the","attention","adapter","as","the","residual","connection",".","Args",":","input",":","A","dictionary","of","multiple","input","arguments","for","the","adapter","module",".","`","query","`",",","`","key","`",",","`","value","`",":","Original","output","tensor","of","the","module",",","or","the","output","of","the","previous","adapter","(","if","more","than","one","adapters","are","enabled",")",".","`","mask","`",":","Attention","mask",".","`","pos_emb","`",":","Optional","positional","embedding","for","relative","encoding",".","adapter",":","The","adapter","module","that","is","currently","required","to","perform","the","forward","pass",".","module",":","The","calling","module",",","in","its","entirety",".","It","is","a","module","that","implements","`","AdapterModuleMixin","`",",","therefore","the","strategy","can","access","all","other","adapters","in","this","module","via","`","module.adapter_layer","`",".","Returns",":","The","result","tensor",",","after","one","of","the","active","adapters","has","finished","its","forward","passes","."]
code: def forward(
      self,
      input: torch.Tensor,
      adapter: torch.nn.Module,
      *,
      module: "AdapterModuleMixin",
  ):
      """
      A basic strategy, comprising of a residual connection over the input, after forward pass by
      the underlying adapter. Additional work is done to pack and unpack the dictionary of inputs and outputs.
  
      Note: The `value` tensor is added to the output of the attention adapter as the residual connection.
  
      Args:
          input: A dictionary of multiple input arguments for the adapter module.
              `query`, `key`, `value`: Original output tensor of the module, or the output of the
               previous adapter (if more than one adapters are enabled).
               `mask`: Attention mask.
               `pos_emb`: Optional positional embedding for relative encoding.
          adapter: The adapter module that is currently required to perform the forward pass.
          module: The calling module, in its entirety. It is a module that implements `AdapterModuleMixin`,
              therefore the strategy can access all other adapters in this module via `module.adapter_layer`.
  
      Returns:
          The result tensor, after one of the active adapters has finished its forward passes.
      """
      out = self.compute_output(input, adapter, module=module)
  
      # If not in training mode, or probability of stochastic depth is 0, skip step.
      p = self.stochastic_depth
      if not module.training or p == 0.0:
          pass
      else:
          out = self.apply_stochastic_depth(
              out, input["value"], adapter, module=module
          )
  
      # Return the residual connection output = input + adapter(input)
      result = input["value"] + out
  
      # If l2_lambda is activated, register the loss value
      self.compute_auxiliary_losses(
          result, input["value"], adapter, module=module
      )
  
      return result
  
imports: import math
  from dataclasses import dataclass, field
  from typing import Any, Optional
  import torch
  from torch import nn
  from nemo.collections.asr.parts.submodules import multi_head_attention
  from nemo.collections.common.parts import adapter_modules
  from nemo.core.classes.mixins import adapter_mixin_strategies
label_desc: Use image node_id 1 for calling the MHAResidualAddAdapterStrategy obj's underlying member method code with example usage: obj.forward(input, adapter) and returns: result