class_docstr: LoRA where single nn.Linear represents more than one layer (used in some implementations of attention query/key/value projections). Weights of linear layer are set to be frozen per default.
  
  Parameters
  ----------
  in_features
      input dimension, set to the original linear layer input dimension LoRA is replacing
  out_features
      output dimension, set to the original linear layer output dimension LoRA is replacing
  r
      rank r of the low-rank decomposition
  lora_alpha
      Scaling factor. Can be simply set to same value as r as initialization is scaled already.
  lora_dropout
      Dropout rate for LoRA
  fan_in_fan_out
      Set this to True if the layer to replace stores weight like (fan_in, fan_out)
  merge_weights
      Merging weights during inference to reduce latency
  
  References
  ----------
  1. Edward J. Hu*, Yelong Shen*, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen,
  "LoRA: Low-Rank Adaptation of Large Language Models", 2021
  https://arxiv.org/abs/2106.09685
  2. Code: https://github.com/microsoft/LoRA
code: def reset_parameters(self):
      nn.Linear.reset_parameters(self)
      if hasattr(self, "lora_A"):
          # initialize A the same way as the default for nn.Linear and B to zero
          nn.init.kaiming_uniform_(self.lora_A, a=math.sqrt(5))
          nn.init.zeros_(self.lora_B)
  
imports: import math
  from typing import List
  import torch
  import torch.nn
  import torch.nn.functional
label_desc: Use image node_id 2 for calling the LoRAMergedLinear obj's underlying member method code with example usage: obj.reset_parameters() without return types