class_docstr: A linear layer that is used with LoRA.
  
  Parameters:
      in_features (`int`):
          Number of input features.
      out_features (`int`):
          Number of output features.
      rank (`int`, `optional`, defaults to 4):
          The rank of the LoRA layer.
      network_alpha (`float`, `optional`, defaults to `None`):
          The value of the network alpha used for stable learning and preventing underflow. This value has the same
          meaning as the `--network_alpha` option in the kohya-ss trainer script. See
          https://github.com/darkstorm2150/sd-scripts/blob/main/docs/train_network_README-en.md#execute-learning
      device (`torch.device`, `optional`, defaults to `None`):
          The device to use for the layer's weights.
      dtype (`torch.dtype`, `optional`, defaults to `None`):
          The dtype to use for the layer's weights.
code: def __init__(
      self,
      in_features: int,
      out_features: int,
      rank: int = 4,
      network_alpha: Optional[float] = None,
      device: Optional[Union[torch.device, str]] = None,
      dtype: Optional[torch.dtype] = None,
  ):
      super().__init__()
  
      self.down = nn.Linear(
          in_features, rank, bias=False, device=device, dtype=dtype
      )
      self.up = nn.Linear(
          rank, out_features, bias=False, device=device, dtype=dtype
      )
      # This value has the same meaning as the `--network_alpha` option in the kohya-ss trainer script.
      # See https://github.com/darkstorm2150/sd-scripts/blob/main/docs/train_network_README-en.md#execute-learning
      self.network_alpha = network_alpha
      self.rank = rank
      self.out_features = out_features
      self.in_features = in_features
  
      nn.init.normal_(self.down.weight, std=1 / rank)
      nn.init.zeros_(self.up.weight)
  
imports: from typing import Optional, Tuple, Union
  import torch
  import torch.nn.functional
  from torch import nn
  from ..utils import logging
  from ..utils.import_utils import is_transformers_available
label_desc: Use image node_id 1 to create a new LoRALinearLayer object from inherited base classes: nn with example: obj = LoRALinearLayer(in_features, out_features, rank, network_alpha, device, dtype)