class_docstr: Container module of the (improved) Transformer proposed in [1].
  
  Reference:
      Dual-path transformer network: Direct context-aware modeling for end-to-end
      monaural speech separation; Chen et al, Interspeech 2020.
  
  Args:
      rnn_type (str): select from 'RNN', 'LSTM' and 'GRU'.
      input_size (int): Dimension of the input feature.
      att_heads (int): Number of attention heads.
      hidden_size (int): Dimension of the hidden state.
      dropout (float): Dropout ratio. Default is 0.
      activation (str): activation function applied at the output of RNN.
      bidirectional (bool, optional): True for bidirectional Inter-Chunk RNN
          (Intra-Chunk is always bidirectional).
      norm (str, optional): Type of normalization to use.
class_docstr_tok: ["Container","module","of","the","(","improved",")","Transformer","proposed","in","[","1","]",".","Reference",":","Dual-path","transformer","network",":","Direct","context-aware","modeling","for","end-to-end","monaural","speech","separation",";","Chen","et","al",",","Interspeech","2020",".","Args",":","rnn_type","(","str",")",":","select","from","'RNN","'",",","'LSTM","'","and","'GRU","'",".","input_size","(","int",")",":","Dimension","of","the","input","feature",".","att_heads","(","int",")",":","Number","of","attention","heads",".","hidden_size","(","int",")",":","Dimension","of","the","hidden","state",".","dropout","(","float",")",":","Dropout","ratio",".","Default","is","0.","activation","(","str",")",":","activation","function","applied","at","the","output","of","RNN",".","bidirectional","(","bool",",","optional",")",":","True","for","bidirectional","Inter-Chunk","RNN","(","Intra-Chunk","is","always","bidirectional",")",".","norm","(","str",",","optional",")",":","Type","of","normalization","to","use","."]
code: def forward(self, x, attn_mask=None):
      # (batch, seq, input_size) -> (seq, batch, input_size)
      src = x.permute(1, 0, 2)
      # (seq, batch, input_size) -> (batch, seq, input_size)
      out = self.self_attn(src, src, src, attn_mask=attn_mask)[
          0
      ].permute(1, 0, 2)
      out = self.dropout(out) + x
      # ... -> (batch, input_size, seq) -> ...
      out = self.norm_attn(out.transpose(-1, -2)).transpose(-1, -2)
  
      out2 = self.feed_forward(self.rnn(out)[0])
      out2 = self.dropout(out2) + out
      return self.norm_ff(out2.transpose(-1, -2)).transpose(-1, -2)
  
file_path: espnet/espnet2/enh/layers/dptnet.py
imports: import torch.nn
  from espnet2.enh.layers.tcn import choose_norm
  from espnet.nets.pytorch_backend.nets_utils import get_activation
label_desc: Use image node_id 2 for calling the ImprovedTransformerLayer obj's underlying member method code with example usage: obj.forward(x, attn_mask) and returns: self