class_docstr: LoRA (low-rank adaptation) followed by (IA)^3 (weight rescaling) incorporated in a Linear Layer. Weights of Linear layer are set to be frozen per default.
  
  Parameters
  ----------
  in_features
      input dimension, set to the original linear layer input dimension LoRA is replacing.
  out_features
      output dimension, set to the original linear layer output dimension LoRA is replacing.
  r
      rank r of the low-rank decomposition.
  lora_alpha
      Scaling factor. Can be simply set to same value as r as initialization is scaled already.
  lora_dropout
      Dropout probability.
  fan_in_fan_out
      Set this to True if the layer to replace stores weight like (fan_in, fan_out).
  merge_weights
      Merging weights during inference to reduce latency.
code: def __init__(
      self,
      in_features: int,
      out_features: int,
      r=8,
      lora_alpha=8,
      lora_dropout: float = 0.0,
      fan_in_fan_out=False,
      merge_weights=False,
      **kwargs,
  ):
      nn.Linear.__init__(self, in_features, out_features, **kwargs)
      LoRALayer.__init__(
          self,
          r=r,
          lora_alpha=lora_alpha,
          lora_dropout=lora_dropout,
          merge_weights=merge_weights,
      )
      # In essence the $b$ parameter of LoRA.
      self.lora_b = nn.Parameter(torch.ones(out_features, 1))
      self.lora_A = nn.Parameter(
          self.weight.new_zeros((r, in_features))
      )
      self.lora_B = nn.Parameter(
          self.weight.new_zeros((out_features, r))
      )
      self.fan_in_fan_out = fan_in_fan_out
      self.weight.requires_grad = False
  
      self.scaling = self.lora_alpha / self.r
      self.reset_parameters()
  
imports: import math
  from typing import List
  import torch
  import torch.nn
  import torch.nn.functional
label_desc: Use image node_id 1 to create a new IA3LoRALinear object from inherited base classes: nn, LoRALayer with example: obj = IA3LoRALinear(in_features, out_features, r, lora_alpha, lora_dropout, fan_in_fan_out, merge_weights)