class_docstr: T5 style feed-forward layer with gated activations and dropout.
  
  Args:
      d_model (`int`):
          Size of the input hidden states.
      d_ff (`int`):
          Size of the intermediate feed-forward layer.
      dropout_rate (`float`):
          Dropout probability.
class_docstr_tok: ["T5","style","feed-forward","layer","with","gated","activations","and","dropout",".","Args",":","d_model","(","`","int","`",")",":","Size","of","the","input","hidden","states",".","d_ff","(","`","int","`",")",":","Size","of","the","intermediate","feed-forward","layer",".","dropout_rate","(","`","float","`",")",":","Dropout","probability","."]
code: def __init__(self, d_model: int, d_ff: int, dropout_rate: float):
      super().__init__()
      self.wi_0 = nn.Linear(d_model, d_ff, bias=False)
      self.wi_1 = nn.Linear(d_model, d_ff, bias=False)
      self.wo = nn.Linear(d_ff, d_model, bias=False)
      self.dropout = nn.Dropout(dropout_rate)
      self.act = NewGELUActivation()
  
imports: import math
  from typing import Optional, Tuple
  import torch
  from torch import nn
  from ..configuration_utils import ConfigMixin, register_to_config
  from .attention_processor import Attention
  from .embeddings import get_timestep_embedding
  from .modeling_utils import ModelMixin
label_desc: Use image node_id 1 to create a new T5DenseGatedActDense object from inherited base classes: nn with example: obj = T5DenseGatedActDense(d_model, d_ff, dropout_rate)