class_docstr: Drop-in replacement for all output linear layers.
  
  An "output" linear layer is one that maps from a width dimension (e.g.,
  `d_model` in a Transformer) to a non-width dimension (e.g., vocab size).
  
  This layer implements the version of μP with a 1/width multiplier and a
  constant variance initialization for both weights and biases.
  Arguments:
      mpu_vocab_size: model parallel size of vocabulary.
      parallel_output: wether output logits being distributed or not.
code: def __init__(self, mpu_vocab_size, parallel_output):
      super(MuReadout, self).__init__()
      self.bias = torch.nn.Parameter(torch.zeros(mpu_vocab_size))
      self.bias.model_parallel = True
      self.bias.partition_dim = 0
      self.bias.stride = 1
      self.parallel_output = parallel_output
      self.warn_once = False
  
file_path: NeMo/nemo/collections/nlp/modules/common/megatron/mup/layer.py
imports: import torch
  from nemo.collections.nlp.modules.common.megatron.module import MegatronModule
  from nemo.collections.nlp.modules.common.megatron.utils import parallel_lm_logits
  from nemo.utils import logging
label_desc: Use image node_id 1 to create a new MuReadout object from inherited base classes: MegatronModule with example: obj = MuReadout(mpu_vocab_size, parallel_output)