class_docstr: LoRA where single nn.Linear represents more than one layer (used in some implementations of attention query/key/value projections). Weights of linear layer are set to be frozen per default.
  
  Parameters
  ----------
  in_features
      input dimension, set to the original linear layer input dimension LoRA is replacing
  out_features
      output dimension, set to the original linear layer output dimension LoRA is replacing
  r
      rank r of the low-rank decomposition
  lora_alpha
      Scaling factor. Can be simply set to same value as r as initialization is scaled already.
  lora_dropout
      Dropout rate for LoRA
  fan_in_fan_out
      Set this to True if the layer to replace stores weight like (fan_in, fan_out)
  merge_weights
      Merging weights during inference to reduce latency
  
  References
  ----------
  1. Edward J. Hu*, Yelong Shen*, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen,
  "LoRA: Low-Rank Adaptation of Large Language Models", 2021
  https://arxiv.org/abs/2106.09685
  2. Code: https://github.com/microsoft/LoRA
code: def zero_pad(self, x):
      result = x.new_zeros((*x.shape[:-1], self.out_features))
      result = result.view(-1, self.out_features)
      result[:, self.lora_ind] = x.reshape(
          -1,
          self.out_features
          // len(self.enable_lora)
          * sum(self.enable_lora),
      )
      return result.view((*x.shape[:-1], self.out_features))
  
imports: import math
  from typing import List
  import torch
  import torch.nn
  import torch.nn.functional
label_desc: Use image node_id 3 for calling the LoRAMergedLinear obj's underlying member method code with example usage: obj.zero_pad(x) and returns: result