class_docstr: LoRA (low-rank adaptation) followed by (IA)^3 (weight rescaling) incorporated in a Linear Layer. Weights of Linear layer are set to be frozen per default.
  
  Parameters
  ----------
  in_features
      input dimension, set to the original linear layer input dimension LoRA is replacing.
  out_features
      output dimension, set to the original linear layer output dimension LoRA is replacing.
  r
      rank r of the low-rank decomposition.
  lora_alpha
      Scaling factor. Can be simply set to same value as r as initialization is scaled already.
  lora_dropout
      Dropout probability.
  fan_in_fan_out
      Set this to True if the layer to replace stores weight like (fan_in, fan_out).
  merge_weights
      Merging weights during inference to reduce latency.
code: def forward(self, x: torch.Tensor):
      result = F.linear(x, self.T(self.weight), bias=self.bias)
      if self.r > 0:
          result += (
              self.lora_dropout(x) @ self.lora_A.T @ self.lora_B.T
          ) * self.scaling
  
      hidden = result * self.lora_b.flatten()
      return hidden
  
imports: import math
  from typing import List
  import torch
  import torch.nn
  import torch.nn.functional
label_desc: Use image node_id 4 for calling the IA3LoRALinear obj's underlying member method code with example usage: obj.forward(x) and returns: hidden