class_docstr: Linear layer with row parallelism.
  
  The linear layer is defined as Y = XA + b. A is parallelized along
  its first dimension and X along its second dimension as:
             -   -
            | A_1 |
            | .   |
        A = | .   |        X = [X_1, ..., X_p]
            | .   |
            | A_p |
             -   -
  Arguments:
      in_features: first dimension of matrix A.
      out_features: second dimension of matrix A.
      bias: If true, add bias. Note that bias is not parallelized.
      input_is_parallel: If true, we assume that the input is already
                         split across the GPUs and we do not split
                         again.
      init_method: method to initialize weights. Note that bias is always set
                   to zero.
      stride: For the strided linear layers.
      keep_master_weight_for_test: This was added for testing and should be
                                   set to False. It returns the master weights
                                   used for initialization.
class_docstr_tok: ["Linear","layer","with","row","parallelism",".","The","linear","layer","is","defined","as","Y","=","XA","+","b",".","A","is","parallelized","along","its","first","dimension","and","X","along","its","second","dimension","as",":","-","-","|","A_1","|","|",".","|","A","=","|",".","|","X","=","[","X_1",",","...",",","X_p","]","|",".","|","|","A_p","|","-","-","Arguments",":","in_features",":","first","dimension","of","matrix","A.","out_features",":","second","dimension","of","matrix","A.","bias",":","If","true",",","add","bias",".","Note","that","bias","is","not","parallelized",".","input_is_parallel",":","If","true",",","we","assume","that","the","input","is","already","split","across","the","GPUs","and","we","do","not","split","again",".","init_method",":","method","to","initialize","weights",".","Note","that","bias","is","always","set","to","zero",".","stride",":","For","the","strided","linear","layers",".","keep_master_weight_for_test",":","This","was","added","for","testing","and","should","be","set","to","False",".","It","returns","the","master","weights","used","for","initialization","."]
code: def forward(
      self, input_: torch.Tensor
  ) -> torch.Tensor:  # type:ignore
      # Set up backprop all-reduce.
      if self.input_is_parallel:
          input_parallel = input_
      else:
          input_parallel = scatter_to_model_parallel_region(input_)
      # Matrix multiply.
      output_parallel = F.linear(input_parallel, self.weight)
      # All-reduce across all the partitions.
      output_ = reduce_from_model_parallel_region(output_parallel)
      if self.bias is not None:
          output = output_ + self.bias
      else:
          output = output_
      return output
  
imports: from typing import Callable, Optional
  import torch
  import torch.nn.functional
  import torch.nn.init
  from torch.nn.parameter import Parameter
  from .initialize import get_model_parallel_rank, get_model_parallel_world_size
  from .mappings import copy_to_model_parallel_region, gather_from_model_parallel_region, reduce_from_model_parallel_region, scatter_to_model_parallel_region
  from .utils import VocabUtility, divide_and_check_no_remainder
label_desc: Use image node_id 3 for calling the RowParallelLinear obj's underlying member method code with example usage: obj.forward(input_) and returns: output