class_docstr: LoRA where single nn.Linear represents more than one layer (used in some implementations of attention query/key/value projections). Weights of linear layer are set to be frozen per default.
  
  Parameters
  ----------
  in_features
      input dimension, set to the original linear layer input dimension LoRA is replacing
  out_features
      output dimension, set to the original linear layer output dimension LoRA is replacing
  r
      rank r of the low-rank decomposition
  lora_alpha
      Scaling factor. Can be simply set to same value as r as initialization is scaled already.
  lora_dropout
      Dropout rate for LoRA
  fan_in_fan_out
      Set this to True if the layer to replace stores weight like (fan_in, fan_out)
  merge_weights
      Merging weights during inference to reduce latency
  
  References
  ----------
  1. Edward J. Hu*, Yelong Shen*, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen,
  "LoRA: Low-Rank Adaptation of Large Language Models", 2021
  https://arxiv.org/abs/2106.09685
  2. Code: https://github.com/microsoft/LoRA
code: def eval(self):
      def T(w):
          return w.T if self.fan_in_fan_out else w
  
      nn.Linear.eval(self)
      if self.merge_weights and not self.merged:
          # Merge the weights and mark it
          if self.r > 0 and any(self.enable_lora):
              delta_w = F.conv1d(
                  self.lora_A.data.unsqueeze(0),
                  self.lora_B.data.unsqueeze(-1),
                  groups=sum(self.enable_lora),
              ).squeeze(0)
              self.weight.data += self.zero_pad(
                  T(delta_w * self.scaling)
              )
          self.merged = True
  
imports: import math
  from typing import List
  import torch
  import torch.nn
  import torch.nn.functional
label_desc: Use image node_id 5 for calling the LoRAMergedLinear obj's underlying member method code with example usage: obj.eval() and returns: unknown