class_docstr: Baseline softmax that does an output linear projection and a softmax.
  
  
      We also support LMCL (Large Margin Cosine Loss) from the CosFace paper. See
      more detailed comment in the MEVO class below.
  
      This is intended to be used with an embedding layer with shared weights.
  
  Args:
      proj_weight (nn.Parameter):
          The shared weight.
      tile_factor (int):
          Unused. It is here to make kernel init easier with MEVO.
      log_softmax (bool):
          If True, use log_softmax instead of softmax.
      margin (float):
          Used in LMCL (when scale != None). See MEVO comments for
          more details.
      scale (Optional[float]):
          Used in LMCL. If scale is None, LMCL is turned off. See
          MEVO comments for more details.
class_docstr_tok: ["Baseline","softmax","that","does","an","output","linear","projection","and","a","softmax",".","We","also","support","LMCL","(","Large","Margin","Cosine","Loss",")","from","the","CosFace","paper",".","See","more","detailed","comment","in","the","MEVO","class","below",".","This","is","intended","to","be","used","with","an","embedding","layer","with","shared","weights",".","Args",":","proj_weight","(","nn.Parameter",")",":","The","shared","weight",".","tile_factor","(","int",")",":","Unused",".","It","is","here","to","make","kernel","init","easier","with","MEVO",".","log_softmax","(","bool",")",":","If","True",",","use","log_softmax","instead","of","softmax",".","margin","(","float",")",":","Used","in","LMCL","(","when","scale","!","=","None",")",".","See","MEVO","comments","for","more","details",".","scale","(","Optional","[","float","]",")",":","Used","in","LMCL",".","If","scale","is","None",",","LMCL","is","turned","off",".","See","MEVO","comments","for","more","details","."]
code: def lmcl_pre_softmax(
      self, input: torch.Tensor, target: torch.Tensor
  ) -> torch.Tensor:
      # normalize feature and fc layer before multiplication
      #   n: number of features (tokens)
      #   k: number of classes (vocab size)
      #   c: hidden dimension (d_model)
      x = F.normalize(input, dim=1)
      w = F.normalize(self.fc.weight, dim=1)
      logits = torch.einsum("nc,kc->nk", x, w)
  
      # add margin
      row_ind = torch.arange(x.shape[0], dtype=torch.long).to(x.device)
      col_ind = target
      logits[row_ind, col_ind] -= self.margin
  
      # add scale
      logits *= self.scale
  
      return logits
  
imports: from typing import Any, Optional, Tuple
  import torch
  from torch import nn
  import torch.distributed
  import torch.nn.functional
label_desc: Use image node_id 2 for calling the BaselineSoftmax obj's underlying member method code with example usage: obj.lmcl_pre_softmax(input, target) and returns: logits