class_docstr: Adapter Mixin that can augment a ModelPT subclass with Adapter support.
  
  This mixin class should be used only with a top level ModelPT subclass.
  This mixin class adds several utility methods which should be subclassed and overriden to
  propagated to the submodules as necessary.
  
  An Adapter module is any Pytorch nn.Module that possess a few properties :
  
  - It's input and output dimension are the same, while the hidden dimension need not be the same.
  - The final layer of the Adapter module is zero-initialized, so that the residual connection to the adapter
      yields the original output.
  
  This mixin adds the following instance variables to the class this inherits it:
  
      -   `adapter_layer`: A torch.nn.ModuleDict(), whose keys are the names of the adapter (globally unique),
              and values are the Adapter nn.Module().
      -   `adapter_cfg`: A OmegaConf DictConfig object that holds the config of the adapters that are initialized.
      -   `adapter_global_cfg_key`: A str representing a key in the model config that can be provided by the user.
          The value resolves to `global_cfg`, and can be overridden via `model.cfg.adapters.global_cfg.*`.
  
  .. note::
  
      This module **is** responsible for maintaining its config. At the ModelPT level, it will access and
      write Adapter config information to `self.cfg.adapters`.
class_docstr_tok: ["Adapter","Mixin","that","can","augment","a","ModelPT","subclass","with","Adapter","support",".","This","mixin","class","should","be","used","only","with","a","top","level","ModelPT","subclass",".","This","mixin","class","adds","several","utility","methods","which","should","be","subclassed","and","overriden","to","propagated","to","the","submodules","as","necessary",".","An","Adapter","module","is","any","Pytorch","nn.Module","that","possess","a","few","properties",":","-","It","'s","input","and","output","dimension","are","the","same",",","while","the","hidden","dimension","need","not","be","the","same",".","-","The","final","layer","of","the","Adapter","module","is","zero-initialized",",","so","that","the","residual","connection","to","the","adapter","yields","the","original","output",".","This","mixin","adds","the","following","instance","variables","to","the","class","this","inherits","it",":","-","`","adapter_layer","`",":","A","torch.nn.ModuleDict","(",")",",","whose","keys","are","the","names","of","the","adapter","(","globally","unique",")",",","and","values","are","the","Adapter","nn.Module","(",")",".","-","`","adapter_cfg","`",":","A","OmegaConf","DictConfig","object","that","holds","the","config","of","the","adapters","that","are","initialized",".","-","`","adapter_global_cfg_key","`",":","A","str","representing","a","key","in","the","model","config","that","can","be","provided","by","the","user",".","The","value","resolves","to","`","global_cfg","`",",","and","can","be","overridden","via","`","model.cfg.adapters.global_cfg.","*","`",".","..","note",":",":","This","module","*","*","is","*","*","responsible","for","maintaining","its","config",".","At","the","ModelPT","level",",","it","will","access","and","write","Adapter","config","information","to","`","self.cfg.adapters","`","."]
docstr: Utility method that restores only the adapter module(s), and not the entire model itself.
  This allows the sharing of adapters which are often just a fraction of the size of the full model,
  enabling easier deliver.
  
  .. note::
  
      During restoration, assumes that the model does not currently already have an adapter with
      the name (if provided), or any adapter that shares a name with the state dict's modules
      (if name is not provided). This is to ensure that each adapter name is globally unique
      in a model.
  
  Args:
      filepath: Filepath of the .pt file.
      name: Optional name of the adapter that will be saved to this file. If None is passed,
          all adapters will be saved to the file. The name must be either the global name (adapter_name),
          or the module level name (module:adapter_name), whichever exactly matches the state dict.
      map_location: Pytorch flag, where to place the adapter(s) state dict(s).
      strict: Pytorch flag, whether to load the weights of the adapter(s) strictly or not.
docstr_tok: ["Utility","method","that","restores","only","the","adapter","module","(","s",")",",","and","not","the","entire","model","itself",".","This","allows","the","sharing","of","adapters","which","are","often","just","a","fraction","of","the","size","of","the","full","model",",","enabling","easier","deliver",".","..","note",":",":","During","restoration",",","assumes","that","the","model","does","not","currently","already","have","an","adapter","with","the","name","(","if","provided",")",",","or","any","adapter","that","shares","a","name","with","the","state","dict","'s","modules","(","if","name","is","not","provided",")",".","This","is","to","ensure","that","each","adapter","name","is","globally","unique","in","a","model",".","Args",":","filepath",":","Filepath","of","the",".pt","file",".","name",":","Optional","name","of","the","adapter","that","will","be","saved","to","this","file",".","If","None","is","passed",",","all","adapters","will","be","saved","to","the","file",".","The","name","must","be","either","the","global","name","(","adapter_name",")",",","or","the","module","level","name","(","module",":","adapter_name",")",",","whichever","exactly","matches","the","state","dict",".","map_location",":","Pytorch","flag",",","where","to","place","the","adapter","(","s",")","state","dict","(","s",")",".","strict",":","Pytorch","flag",",","whether","to","load","the","weights","of","the","adapter","(","s",")","strictly","or","not","."]
code: def load_adapters(
      self,
      filepath: str,
      name: str = None,
      map_location: str = None,
      strict: bool = True,
  ):
      """
      Utility method that restores only the adapter module(s), and not the entire model itself.
      This allows the sharing of adapters which are often just a fraction of the size of the full model,
      enabling easier deliver.
  
      .. note::
  
          During restoration, assumes that the model does not currently already have an adapter with
          the name (if provided), or any adapter that shares a name with the state dict's modules
          (if name is not provided). This is to ensure that each adapter name is globally unique
          in a model.
  
      Args:
          filepath: Filepath of the .pt file.
          name: Optional name of the adapter that will be saved to this file. If None is passed,
              all adapters will be saved to the file. The name must be either the global name (adapter_name),
              or the module level name (module:adapter_name), whichever exactly matches the state dict.
          map_location: Pytorch flag, where to place the adapter(s) state dict(s).
          strict: Pytorch flag, whether to load the weights of the adapter(s) strictly or not.
      """
      # Determine device
      if map_location is None:
          if torch.cuda.is_available():
              map_location = "cuda"
          else:
              map_location = "cpu"
  
      # Load the state dict and extract the internal config
      state_dict = torch.load(filepath, map_location=map_location)
      config = state_dict.pop("__cfg__")
  
      # Normalize the name to a list of names (exact match with the state dict)
      if isinstance(name, str):
          name = [name]
  
      if name is None:
          name = list(config.keys())
  
      # For all module:adapter names (note, for global modules, we ignore the module: part)
      for module_adapter_name in name:
          # Extract current config as copy
          internal_adapter_cfg = None
          if (
              hasattr(self, "adapter_cfg")
              and self.adapter_cfg is not None
          ):
              internal_adapter_cfg = self.adapter_cfg
  
          # Override internal adapter config with restoration config
          self.adapter_cfg = config
  
          # Resolve the adapter name and extract the adapter's config from the checkpoint.
          module_name, adapter_name = self.resolve_adapter_module_name_(
              module_adapter_name
          )
          adapter_cfg = config[adapter_name]
  
          # Recreate the module:adapter_name
          if module_name == "":
              module_adapter_name = adapter_name
          else:
              module_adapter_name = f"{module_name}:{adapter_name}"
  
          # Reset internal adapter config
          self.adapter_cfg = internal_adapter_cfg
  
          # Skip the global config key
          if adapter_name == self.adapter_global_cfg_key:
              continue
  
          # Restore weights with exact key, if it fails, give useful error message.
          try:
              adapter_state = state_dict[module_adapter_name]
          except KeyError:
              all_keys = list(state_dict.keys())
              raise KeyError(
                  f"Requested to load adapter with name `{module_adapter_name}`, but could not "
                  f"the adapter in the state dict. \nAvailable adapter names in state dict are: "
                  f"{all_keys}"
              )
  
          # If key was found, add a new adapter with random weights
          self.add_adapter(name=module_adapter_name, cfg=adapter_cfg)
  
          # Determine apriori how many modules must be loaded from the state dict
          # This is dont to guarentee that partial match does not occur, only exact match
          # between state dict and the adapters parameters will be allowed.
          modules_to_load = []  # type: List[torch.nn.Module]
          for module in self.modules():
              if isinstance(module, AdapterModuleMixin):
                  adapter_module = module.get_adapter_module(
                      adapter_name
                  )
                  if adapter_module is not None:
                      modules_to_load.append(adapter_module)
  
          # Assert that the number of states in the state dict matches the newly created adapter
          if len(adapter_state) != len(modules_to_load):
              raise ValueError(
                  f"The number of adapters in current model ({len(modules_to_load)}) does not "
                  f"match the number of modules in the state dict for adapter `{adapter_name}`: "
                  f"({len(adapter_state)})"
              )
  
          # For the pair of (adapter_state_in_checkpoint, adapter_in_model), restore the weights
          for state, module in zip(adapter_state, modules_to_load):
              # Note that state is a list of multiple state dicts for 1:1 Module mapping.
              # However, the state_dict keys are of the form `adapter_name.<module hierarchy with dots>`.
              # We therefore strip the `adapter_name.` part of the state dict
              # And then directly load each module with its 1:1 state dict.
              sub_dict = {}
              for k, v in state.items():
                  if adapter_name in k:
                      k_ = k.replace(f"{adapter_name}.", "")
                      sub_dict[k_] = v
  
              module.load_state_dict(sub_dict, strict=strict)
              del sub_dict
  
          # delete the dictionaries to preserve memory for next adapter
          del adapter_state, modules_to_load
  
imports: import inspect
  from abc import ABC
  from dataclasses import dataclass, is_dataclass
  from typing import List, Optional, Set, Tuple, Union
  import torch
  import torch.nn
  from hydra.utils import instantiate
  from omegaconf import DictConfig, OmegaConf, open_dict
  from nemo.utils import logging, model_utils
label_desc: Use image node_id 8 for calling the AdapterModelPTMixin obj's underlying member method code with example usage: obj.load_adapters(filepath, name, map_location, strict) without return types