class_docstr: A wrapper class around a (`transformers.PreTrainedModel`) to be compatible with the
  (`~transformers.PreTrained`) class in order to keep some attributes and methods of the
  (`~transformers.PreTrainedModel`) class.
  
  Attributes:
      pretrained_model: (`transformers.PreTrainedModel`)
          The model to be wrapped.
      parent_class: (`transformers.PreTrainedModel`)
          The parent class of the model to be wrapped.
      supported_args: (`list`)
          The list of arguments that are supported by the wrapper class.
class_docstr_tok: ["A","wrapper","class","around","a","(","`","transformers.PreTrainedModel","`",")","to","be","compatible","with","the","(","`","~transformers.PreTrained","`",")","class","in","order","to","keep","some","attributes","and","methods","of","the","(","`","~transformers.PreTrainedModel","`",")","class",".","Attributes",":","pretrained_model",":","(","`","transformers.PreTrainedModel","`",")","The","model","to","be","wrapped",".","parent_class",":","(","`","transformers.PreTrainedModel","`",")","The","parent","class","of","the","model","to","be","wrapped",".","supported_args",":","(","`","list","`",")","The","list","of","arguments","that","are","supported","by","the","wrapper","class","."]
docstr: Add and load a reward modeling adapter. This method can only be used if the
  model is a `PeftModel` and if you have initialized the model with the `reward_modeling_adapter_id`
  argument, pointing to the id of the reward modeling adapter. The latest needs also to contain the
  score head in order to produce the reward.
docstr_tok: ["Add","and","load","a","reward","modeling","adapter",".","This","method","can","only","be","used","if","the","model","is","a","`","PeftModel","`","and","if","you","have","initialized","the","model","with","the","`","reward_modeling_adapter_id","`","argument",",","pointing","to","the","id","of","the","reward","modeling","adapter",".","The","latest","needs","also","to","contain","the","score","head","in","order","to","produce","the","reward","."]
code: def add_and_load_reward_modeling_adapter(
      cls,
      pretrained_model,
      adapter_model_id,
      adapter_name="reward_model_adapter",
      token=None,
  ):
      r"""
      Add and load a reward modeling adapter. This method can only be used if the
      model is a `PeftModel` and if you have initialized the model with the `reward_modeling_adapter_id`
      argument, pointing to the id of the reward modeling adapter. The latest needs also to contain the
      score head in order to produce the reward.
      """
      pretrained_model.load_adapter(
          adapter_model_id, adapter_name, is_trainable=False
      )
      pretrained_model.train()
  
      filename = os.path.join(adapter_model_id, "adapter_model.bin")
      safe_loading = False
      if not os.path.exists(filename):
          try:
              local_filename = hf_hub_download(
                  adapter_model_id,
                  "adapter_model.bin",
                  token=token,
              )
          except:  # noqa
              filename = os.path.join(
                  adapter_model_id, "adapter_model.safetensors"
              )
              safe_loading = True
              if not os.path.exists(filename):
                  try:
                      local_filename = hf_hub_download(
                          adapter_model_id,
                          "adapter_model.safetensors",
                          token=token,
                      )
                  except:  # noqa
                      raise ValueError(
                          "Could not find adapter model in the Hub, make sure you have the correct adapter model id."
                      )
              else:
                  local_filename = filename
      else:
          local_filename = filename
  
      loading_func = safe_load_file if safe_loading else torch.load
      load_kwargs = {} if safe_loading else {"map_location": "cpu"}
  
      adapter_state_dict = loading_func(local_filename, **load_kwargs)
  
      for score_name_candidate in cls.supported_rm_modules:
          if any(
              [
                  score_name_candidate in name
                  for name in adapter_state_dict.keys()
              ]
          ):
              score_name = score_name_candidate
              # we have found the correct head name and can break
              break
  
      score_dict = {}
  
      for name, param in adapter_state_dict.items():
          if score_name in name:
              key_name = ".".join(name.split(".")[-1:])
              score_dict[key_name] = param.to(cls._get_current_device())
  
      num_labels, hidden_dim = score_dict["weight"].shape
      has_bias = any(
          ["bias" in name for name in adapter_state_dict.keys()]
      )
  
      score = nn.Linear(hidden_dim, num_labels, bias=has_bias).to(
          device=cls._get_current_device(),
          dtype=pretrained_model.dtype,
      )
      score.load_state_dict(score_dict)
      for param in score.parameters():
          param.requires_grad = False
  
      return score
  
imports: import json
  import logging
  import os
  from copy import deepcopy
  import torch
  import torch.nn
  from accelerate import PartialState
  from huggingface_hub import hf_hub_download
  from huggingface_hub.utils import EntryNotFoundError, HFValidationError, LocalEntryNotFoundError, RepositoryNotFoundError
  from safetensors.torch import load_file
  from transformers import PreTrainedModel
  from ..import_utils import is_npu_available, is_peft_available, is_transformers_greater_than, is_xpu_available
label_desc: Use image node_id 6 for calling the PreTrainedModelWrapper obj's underlying member method code with example usage: obj.add_and_load_reward_modeling_adapter(cls, pretrained_model, adapter_model_id, adapter_name, token) and returns: score