class_docstr: LoRA incorporated in Embedding Layer. Weights of embedding layer are set to be frozen per default.
  
  Parameters
  ----------
  num_embeddings
      size of the dictionary of embeddings.
  embedding_dim
       the size of each embedding vector.
  r
      rank r of the low-rank decomposition.
  lora_alpha
      Scaling factor. Can be simply set to same value as r as initialization is scaled already.
  merge_weights
      Merging weights during inference to reduce latency.
  
  References
  ----------
  1. Edward J. Hu*, Yelong Shen*, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen,
  "LoRA: Low-Rank Adaptation of Large Language Models", 2021
  https://arxiv.org/abs/2106.09685
  2. Code: https://github.com/microsoft/LoRA
code: def __init__(
      self,
      num_embeddings: int,
      embedding_dim: int,
      r: int = 0,
      lora_alpha: int = 1,
      merge_weights: bool = True,
      **kwargs,
  ):
      nn.Embedding.__init__(
          self, num_embeddings, embedding_dim, **kwargs
      )
      LoRALayer.__init__(
          self,
          r=r,
          lora_alpha=lora_alpha,
          lora_dropout=0,
          merge_weights=merge_weights,
      )
      # Actual trainable parameters
      if r > 0:
          self.lora_A = nn.Parameter(
              self.weight.new_zeros((r, num_embeddings))
          )
          self.lora_B = nn.Parameter(
              self.weight.new_zeros((embedding_dim, r))
          )
          self.scaling = self.lora_alpha / self.r
          # Freezing the pre-trained weight matrix
          self.weight.requires_grad = False
      self.reset_parameters()
  
imports: import math
  from typing import List
  import torch
  import torch.nn
  import torch.nn.functional
label_desc: Use image node_id 1 to create a new LoRAEmbedding object from inherited base classes: nn, LoRALayer with example: obj = LoRAEmbedding(num_embeddings, embedding_dim, r, lora_alpha, merge_weights)