class_docstr: Position-wise feed-forward network of Transformer block.
  
  Args:
      hidden_size: size of the embeddings in the model, also known as d_model
      inner_size: number of neurons in the intermediate part of feed-forward
          net, usually is (4-8 x hidden_size) in the papers
      ffn_dropout: probability of dropout applied to net output
      hidden_act: activation function used between two linear layers
class_docstr_tok: ["Position-wise","feed-forward","network","of","Transformer","block",".","Args",":","hidden_size",":","size","of","the","embeddings","in","the","model",",","also","known","as","d_model","inner_size",":","number","of","neurons","in","the","intermediate","part","of","feed-forward","net",",","usually","is","(","4-8","x","hidden_size",")","in","the","papers","ffn_dropout",":","probability","of","dropout","applied","to","net","output","hidden_act",":","activation","function","used","between","two","linear","layers"]
code: def __init__(
      self, hidden_size, inner_size, ffn_dropout=0.0, hidden_act="relu"
  ):
      super().__init__()
      self.dense_in = nn.Linear(hidden_size, inner_size)
      self.dense_out = nn.Linear(inner_size, hidden_size)
      self.layer_dropout = nn.Dropout(ffn_dropout)
      ACT2FN = {"gelu": gelu, "relu": torch.relu}
      self.act_fn = ACT2FN[hidden_act]
  
imports: import math
  import numpy
  import torch
  from torch import nn
  from torch.nn.functional import gelu
  from nemo.collections.common.parts import form_attention_mask
  from nemo.utils import logging
label_desc: Use image node_id 1 to create a new PositionWiseFF object from inherited base classes: nn with example: obj = PositionWiseFF(hidden_size, inner_size, ffn_dropout, hidden_act)