class_docstr: Implements the FFN part of the transformer.
code: def __init__(
      self,
      opt: Opt,
      dim: int = None,
      dim_hidden: int = None,
      relu_dropout: float = 0,
      activation: str = "relu",
      **kwargs,
  ):
      super(TransformerFFN, self).__init__(**kwargs)
  
      dim = default(dim, opt["embedding_size"])
      dim_hidden = default(dim_hidden, opt["ffn_size"])
  
      self.opt = opt
      self.relu_dropout = nn.Dropout(p=relu_dropout)
      if activation == "relu":
          self.nonlinear = F.relu
      elif activation == "gelu":
          self.nonlinear = F.gelu
      else:
          raise ValueError(
              "Don't know how to handle --activation {}".format(
                  activation
              )
          )
      self.lin1 = nn.Linear(dim, dim_hidden)
      self.lin2 = nn.Linear(dim_hidden, dim)
      nn.init.xavier_uniform_(self.lin1.weight)
      nn.init.xavier_uniform_(self.lin2.weight)
  
file_path: ParlAI/parlai/agents/transformer/modules/ffn.py
imports: import torch
  import torch.nn
  import torch.nn.functional
  from parlai.core.opt import Opt
  from parlai.core.params import default
label_desc: Use image node_id 1 to create a new TransformerFFN object from inherited base classes: nn with example: obj = TransformerFFN(opt, dim, dim_hidden, relu_dropout, activation)