class_docstr: LoRA (low-rank adaptation) followed by (IA)^3 (weight rescaling) incorporated in a Linear Layer. Weights of Linear layer are set to be frozen per default.
  
  Parameters
  ----------
  in_features
      input dimension, set to the original linear layer input dimension LoRA is replacing.
  out_features
      output dimension, set to the original linear layer output dimension LoRA is replacing.
  r
      rank r of the low-rank decomposition.
  lora_alpha
      Scaling factor. Can be simply set to same value as r as initialization is scaled already.
  lora_dropout
      Dropout probability.
  fan_in_fan_out
      Set this to True if the layer to replace stores weight like (fan_in, fan_out).
  merge_weights
      Merging weights during inference to reduce latency.
code: def eval(self):
      # def T(w):
      #     return w.T if self.fan_in_fan_out else w
      nn.Linear.eval(self)
      if self.merge_weights and not self.merged:
          # Merge the weights and mark it
          if self.r > 0:
              self.weight.data += (
                  self.T(self.lora_B @ self.lora_A) * self.scaling
              )
              self.weight.data *= self.lora_b.flatten()
          self.merged = True
      return hidden
  
imports: import math
  from typing import List
  import torch
  import torch.nn
  import torch.nn.functional
label_desc: Use image node_id 6 for calling the IA3LoRALinear obj's underlying member method code with example usage: obj.eval() and returns: hidden