class_docstr: Container module of the (improved) Transformer proposed in [1].
  
  Reference:
      Dual-path transformer network: Direct context-aware modeling for end-to-end
      monaural speech separation; Chen et al, Interspeech 2020.
  
  Args:
      rnn_type (str): select from 'RNN', 'LSTM' and 'GRU'.
      input_size (int): Dimension of the input feature.
      att_heads (int): Number of attention heads.
      hidden_size (int): Dimension of the hidden state.
      dropout (float): Dropout ratio. Default is 0.
      activation (str): activation function applied at the output of RNN.
      bidirectional (bool, optional): True for bidirectional Inter-Chunk RNN
          (Intra-Chunk is always bidirectional).
      norm (str, optional): Type of normalization to use.
class_docstr_tok: ["Container","module","of","the","(","improved",")","Transformer","proposed","in","[","1","]",".","Reference",":","Dual-path","transformer","network",":","Direct","context-aware","modeling","for","end-to-end","monaural","speech","separation",";","Chen","et","al",",","Interspeech","2020",".","Args",":","rnn_type","(","str",")",":","select","from","'RNN","'",",","'LSTM","'","and","'GRU","'",".","input_size","(","int",")",":","Dimension","of","the","input","feature",".","att_heads","(","int",")",":","Number","of","attention","heads",".","hidden_size","(","int",")",":","Dimension","of","the","hidden","state",".","dropout","(","float",")",":","Dropout","ratio",".","Default","is","0.","activation","(","str",")",":","activation","function","applied","at","the","output","of","RNN",".","bidirectional","(","bool",",","optional",")",":","True","for","bidirectional","Inter-Chunk","RNN","(","Intra-Chunk","is","always","bidirectional",")",".","norm","(","str",",","optional",")",":","Type","of","normalization","to","use","."]
code: def __init__(
      self,
      rnn_type,
      input_size,
      att_heads,
      hidden_size,
      dropout=0.0,
      activation="relu",
      bidirectional=True,
      norm="gLN",
  ):
      super().__init__()
  
      rnn_type = rnn_type.upper()
      assert rnn_type in [
          "RNN",
          "LSTM",
          "GRU",
      ], f"Only support 'RNN', 'LSTM' and 'GRU', current type: {rnn_type}"
      self.rnn_type = rnn_type
  
      self.att_heads = att_heads
      self.self_attn = nn.MultiheadAttention(
          input_size, att_heads, dropout=dropout
      )
      self.dropout = nn.Dropout(p=dropout)
      self.norm_attn = choose_norm(norm, input_size)
  
      self.rnn = getattr(nn, rnn_type)(
          input_size,
          hidden_size,
          1,
          batch_first=True,
          bidirectional=bidirectional,
      )
  
      hdim = 2 * hidden_size if bidirectional else hidden_size
      if activation.lower() == "linear":
          activation = nn.Identity()
      else:
          activation = get_activation(activation)
      self.feed_forward = nn.Sequential(
          activation, nn.Dropout(p=dropout), nn.Linear(hdim, input_size)
      )
  
      self.norm_ff = choose_norm(norm, input_size)
  
file_path: espnet/espnet2/enh/layers/dptnet.py
imports: import torch.nn
  from espnet2.enh.layers.tcn import choose_norm
  from espnet.nets.pytorch_backend.nets_utils import get_activation
label_desc: Use image node_id 1 to create a new ImprovedTransformerLayer object from inherited base classes: nn with example: obj = ImprovedTransformerLayer(rnn_type, input_size, att_heads, hidden_size, dropout, activation, bidirectional, norm)