class_docstr: Transformer-TTS module.
  
  This is a module of text-to-speech Transformer described in `Neural Speech Synthesis
  with Transformer Network`_, which convert the sequence of tokens into the sequence
  of Mel-filterbanks.
  
  .. _`Neural Speech Synthesis with Transformer Network`:
      https://arxiv.org/pdf/1809.08895.pdf
docstr: Initialize Transformer module.
  
  Args:
      idim (int): Dimension of the inputs.
      odim (int): Dimension of the outputs.
      embed_dim (int): Dimension of character embedding.
      eprenet_conv_layers (int): Number of encoder prenet convolution layers.
      eprenet_conv_chans (int): Number of encoder prenet convolution channels.
      eprenet_conv_filts (int): Filter size of encoder prenet convolution.
      dprenet_layers (int): Number of decoder prenet layers.
      dprenet_units (int): Number of decoder prenet hidden units.
      elayers (int): Number of encoder layers.
      eunits (int): Number of encoder hidden units.
      adim (int): Number of attention transformation dimensions.
      aheads (int): Number of heads for multi head attention.
      dlayers (int): Number of decoder layers.
      dunits (int): Number of decoder hidden units.
      postnet_layers (int): Number of postnet layers.
      postnet_chans (int): Number of postnet channels.
      postnet_filts (int): Filter size of postnet.
      use_scaled_pos_enc (bool): Whether to use trainable scaled pos encoding.
      use_batch_norm (bool): Whether to use batch normalization in encoder prenet.
      encoder_normalize_before (bool): Whether to apply layernorm layer before
          encoder block.
      decoder_normalize_before (bool): Whether to apply layernorm layer before
          decoder block.
      encoder_concat_after (bool): Whether to concatenate attention layer's input
          and output in encoder.
      decoder_concat_after (bool): Whether to concatenate attention layer's input
          and output in decoder.
      positionwise_layer_type (str): Position-wise operation type.
      positionwise_conv_kernel_size (int): Kernel size in position wise conv 1d.
      reduction_factor (int): Reduction factor.
      spks (Optional[int]): Number of speakers. If set to > 1, assume that the
          sids will be provided as the input and use sid embedding layer.
      langs (Optional[int]): Number of languages. If set to > 1, assume that the
          lids will be provided as the input and use sid embedding layer.
      spk_embed_dim (Optional[int]): Speaker embedding dimension. If set to > 0,
          assume that spembs will be provided as the input.
      spk_embed_integration_type (str): How to integrate speaker embedding.
      use_gst (str): Whether to use global style token.
      gst_tokens (int): Number of GST embeddings.
      gst_heads (int): Number of heads in GST multihead attention.
      gst_conv_layers (int): Number of conv layers in GST.
      gst_conv_chans_list: (Sequence[int]): List of the number of channels of conv
          layers in GST.
      gst_conv_kernel_size (int): Kernel size of conv layers in GST.
      gst_conv_stride (int): Stride size of conv layers in GST.
      gst_gru_layers (int): Number of GRU layers in GST.
      gst_gru_units (int): Number of GRU units in GST.
      transformer_lr (float): Initial value of learning rate.
      transformer_warmup_steps (int): Optimizer warmup steps.
      transformer_enc_dropout_rate (float): Dropout rate in encoder except
          attention and positional encoding.
      transformer_enc_positional_dropout_rate (float): Dropout rate after encoder
          positional encoding.
      transformer_enc_attn_dropout_rate (float): Dropout rate in encoder
          self-attention module.
      transformer_dec_dropout_rate (float): Dropout rate in decoder except
          attention & positional encoding.
      transformer_dec_positional_dropout_rate (float): Dropout rate after decoder
          positional encoding.
      transformer_dec_attn_dropout_rate (float): Dropout rate in decoder
          self-attention module.
      transformer_enc_dec_attn_dropout_rate (float): Dropout rate in source
          attention module.
      init_type (str): How to initialize transformer parameters.
      init_enc_alpha (float): Initial value of alpha in scaled pos encoding of the
          encoder.
      init_dec_alpha (float): Initial value of alpha in scaled pos encoding of the
          decoder.
      eprenet_dropout_rate (float): Dropout rate in encoder prenet.
      dprenet_dropout_rate (float): Dropout rate in decoder prenet.
      postnet_dropout_rate (float): Dropout rate in postnet.
      use_masking (bool): Whether to apply masking for padded part in loss
          calculation.
      use_weighted_masking (bool): Whether to apply weighted masking in loss
          calculation.
      bce_pos_weight (float): Positive sample weight in bce calculation
          (only for use_masking=true).
      loss_type (str): How to calculate loss.
      use_guided_attn_loss (bool): Whether to use guided attention loss.
      num_heads_applied_guided_attn (int): Number of heads in each layer to apply
          guided attention loss.
      num_layers_applied_guided_attn (int): Number of layers to apply guided
          attention loss.
      modules_applied_guided_attn (Sequence[str]): List of module names to apply
          guided attention loss.
      guided_attn_loss_sigma (float) Sigma in guided attention loss.
      guided_attn_loss_lambda (float): Lambda in guided attention loss.
docstr_tok: ["Initialize","Transformer","module",".","Args",":","idim","(","int",")",":","Dimension","of","the","inputs",".","odim","(","int",")",":","Dimension","of","the","outputs",".","embed_dim","(","int",")",":","Dimension","of","character","embedding",".","eprenet_conv_layers","(","int",")",":","Number","of","encoder","prenet","convolution","layers",".","eprenet_conv_chans","(","int",")",":","Number","of","encoder","prenet","convolution","channels",".","eprenet_conv_filts","(","int",")",":","Filter","size","of","encoder","prenet","convolution",".","dprenet_layers","(","int",")",":","Number","of","decoder","prenet","layers",".","dprenet_units","(","int",")",":","Number","of","decoder","prenet","hidden","units",".","elayers","(","int",")",":","Number","of","encoder","layers",".","eunits","(","int",")",":","Number","of","encoder","hidden","units",".","adim","(","int",")",":","Number","of","attention","transformation","dimensions",".","aheads","(","int",")",":","Number","of","heads","for","multi","head","attention",".","dlayers","(","int",")",":","Number","of","decoder","layers",".","dunits","(","int",")",":","Number","of","decoder","hidden","units",".","postnet_layers","(","int",")",":","Number","of","postnet","layers",".","postnet_chans","(","int",")",":","Number","of","postnet","channels",".","postnet_filts","(","int",")",":","Filter","size","of","postnet",".","use_scaled_pos_enc","(","bool",")",":","Whether","to","use","trainable","scaled","pos","encoding",".","use_batch_norm","(","bool",")",":","Whether","to","use","batch","normalization","in","encoder","prenet",".","encoder_normalize_before","(","bool",")",":","Whether","to","apply","layernorm","layer","before","encoder","block",".","decoder_normalize_before","(","bool",")",":","Whether","to","apply","layernorm","layer","before","decoder","block",".","encoder_concat_after","(","bool",")",":","Whether","to","concatenate","attention","layer","'s","input","and","output","in","encoder",".","decoder_concat_after","(","bool",")",":","Whether","to","concatenate","attention","layer","'s","input","and","output","in","decoder",".","positionwise_layer_type","(","str",")",":","Position-wise","operation","type",".","positionwise_conv_kernel_size","(","int",")",":","Kernel","size","in","position","wise","conv","1d",".","reduction_factor","(","int",")",":","Reduction","factor",".","spks","(","Optional","[","int","]",")",":","Number","of","speakers",".","If","set","to",">","1",",","assume","that","the","sids","will","be","provided","as","the","input","and","use","sid","embedding","layer",".","langs","(","Optional","[","int","]",")",":","Number","of","languages",".","If","set","to",">","1",",","assume","that","the","lids","will","be","provided","as","the","input","and","use","sid","embedding","layer",".","spk_embed_dim","(","Optional","[","int","]",")",":","Speaker","embedding","dimension",".","If","set","to",">","0",",","assume","that","spembs","will","be","provided","as","the","input",".","spk_embed_integration_type","(","str",")",":","How","to","integrate","speaker","embedding",".","use_gst","(","str",")",":","Whether","to","use","global","style","token",".","gst_tokens","(","int",")",":","Number","of","GST","embeddings",".","gst_heads","(","int",")",":","Number","of","heads","in","GST","multihead","attention",".","gst_conv_layers","(","int",")",":","Number","of","conv","layers","in","GST",".","gst_conv_chans_list",":","(","Sequence","[","int","]",")",":","List","of","the","number","of","channels","of","conv","layers","in","GST",".","gst_conv_kernel_size","(","int",")",":","Kernel","size","of","conv","layers","in","GST",".","gst_conv_stride","(","int",")",":","Stride","size","of","conv","layers","in","GST",".","gst_gru_layers","(","int",")",":","Number","of","GRU","layers","in","GST",".","gst_gru_units","(","int",")",":","Number","of","GRU","units","in","GST",".","transformer_lr","(","float",")",":","Initial","value","of","learning","rate",".","transformer_warmup_steps","(","int",")",":","Optimizer","warmup","steps",".","transformer_enc_dropout_rate","(","float",")",":","Dropout","rate","in","encoder","except","attention","and","positional","encoding",".","transformer_enc_positional_dropout_rate","(","float",")",":","Dropout","rate","after","encoder","positional","encoding",".","transformer_enc_attn_dropout_rate","(","float",")",":","Dropout","rate","in","encoder","self-attention","module",".","transformer_dec_dropout_rate","(","float",")",":","Dropout","rate","in","decoder","except","attention","&","positional","encoding",".","transformer_dec_positional_dropout_rate","(","float",")",":","Dropout","rate","after","decoder","positional","encoding",".","transformer_dec_attn_dropout_rate","(","float",")",":","Dropout","rate","in","decoder","self-attention","module",".","transformer_enc_dec_attn_dropout_rate","(","float",")",":","Dropout","rate","in","source","attention","module",".","init_type","(","str",")",":","How","to","initialize","transformer","parameters",".","init_enc_alpha","(","float",")",":","Initial","value","of","alpha","in","scaled","pos","encoding","of","the","encoder",".","init_dec_alpha","(","float",")",":","Initial","value","of","alpha","in","scaled","pos","encoding","of","the","decoder",".","eprenet_dropout_rate","(","float",")",":","Dropout","rate","in","encoder","prenet",".","dprenet_dropout_rate","(","float",")",":","Dropout","rate","in","decoder","prenet",".","postnet_dropout_rate","(","float",")",":","Dropout","rate","in","postnet",".","use_masking","(","bool",")",":","Whether","to","apply","masking","for","padded","part","in","loss","calculation",".","use_weighted_masking","(","bool",")",":","Whether","to","apply","weighted","masking","in","loss","calculation",".","bce_pos_weight","(","float",")",":","Positive","sample","weight","in","bce","calculation","(","only","for","use_masking=true",")",".","loss_type","(","str",")",":","How","to","calculate","loss",".","use_guided_attn_loss","(","bool",")",":","Whether","to","use","guided","attention","loss",".","num_heads_applied_guided_attn","(","int",")",":","Number","of","heads","in","each","layer","to","apply","guided","attention","loss",".","num_layers_applied_guided_attn","(","int",")",":","Number","of","layers","to","apply","guided","attention","loss",".","modules_applied_guided_attn","(","Sequence","[","str","]",")",":","List","of","module","names","to","apply","guided","attention","loss",".","guided_attn_loss_sigma","(","float",")","Sigma","in","guided","attention","loss",".","guided_attn_loss_lambda","(","float",")",":","Lambda","in","guided","attention","loss","."]
code: def __init__(
      self,
      # network structure related
      idim: int,
      odim: int,
      embed_dim: int = 512,
      eprenet_conv_layers: int = 3,
      eprenet_conv_chans: int = 256,
      eprenet_conv_filts: int = 5,
      dprenet_layers: int = 2,
      dprenet_units: int = 256,
      elayers: int = 6,
      eunits: int = 1024,
      adim: int = 512,
      aheads: int = 4,
      dlayers: int = 6,
      dunits: int = 1024,
      postnet_layers: int = 5,
      postnet_chans: int = 256,
      postnet_filts: int = 5,
      positionwise_layer_type: str = "conv1d",
      positionwise_conv_kernel_size: int = 1,
      use_scaled_pos_enc: bool = True,
      use_batch_norm: bool = True,
      encoder_normalize_before: bool = True,
      decoder_normalize_before: bool = True,
      encoder_concat_after: bool = False,
      decoder_concat_after: bool = False,
      reduction_factor: int = 1,
      # extra embedding related
      spks: Optional[int] = None,
      langs: Optional[int] = None,
      spk_embed_dim: Optional[int] = None,
      spk_embed_integration_type: str = "add",
      use_gst: bool = False,
      gst_tokens: int = 10,
      gst_heads: int = 4,
      gst_conv_layers: int = 6,
      gst_conv_chans_list: Sequence[int] = (32, 32, 64, 64, 128, 128),
      gst_conv_kernel_size: int = 3,
      gst_conv_stride: int = 2,
      gst_gru_layers: int = 1,
      gst_gru_units: int = 128,
      # training related
      transformer_enc_dropout_rate: float = 0.1,
      transformer_enc_positional_dropout_rate: float = 0.1,
      transformer_enc_attn_dropout_rate: float = 0.1,
      transformer_dec_dropout_rate: float = 0.1,
      transformer_dec_positional_dropout_rate: float = 0.1,
      transformer_dec_attn_dropout_rate: float = 0.1,
      transformer_enc_dec_attn_dropout_rate: float = 0.1,
      eprenet_dropout_rate: float = 0.5,
      dprenet_dropout_rate: float = 0.5,
      postnet_dropout_rate: float = 0.5,
      init_type: str = "xavier_uniform",
      init_enc_alpha: float = 1.0,
      init_dec_alpha: float = 1.0,
      use_masking: bool = False,
      use_weighted_masking: bool = False,
      bce_pos_weight: float = 5.0,
      loss_type: str = "L1",
      use_guided_attn_loss: bool = True,
      num_heads_applied_guided_attn: int = 2,
      num_layers_applied_guided_attn: int = 2,
      modules_applied_guided_attn: Sequence[str] = ("encoder-decoder"),
      guided_attn_loss_sigma: float = 0.4,
      guided_attn_loss_lambda: float = 1.0,
  ):
      """Initialize Transformer module.
  
      Args:
          idim (int): Dimension of the inputs.
          odim (int): Dimension of the outputs.
          embed_dim (int): Dimension of character embedding.
          eprenet_conv_layers (int): Number of encoder prenet convolution layers.
          eprenet_conv_chans (int): Number of encoder prenet convolution channels.
          eprenet_conv_filts (int): Filter size of encoder prenet convolution.
          dprenet_layers (int): Number of decoder prenet layers.
          dprenet_units (int): Number of decoder prenet hidden units.
          elayers (int): Number of encoder layers.
          eunits (int): Number of encoder hidden units.
          adim (int): Number of attention transformation dimensions.
          aheads (int): Number of heads for multi head attention.
          dlayers (int): Number of decoder layers.
          dunits (int): Number of decoder hidden units.
          postnet_layers (int): Number of postnet layers.
          postnet_chans (int): Number of postnet channels.
          postnet_filts (int): Filter size of postnet.
          use_scaled_pos_enc (bool): Whether to use trainable scaled pos encoding.
          use_batch_norm (bool): Whether to use batch normalization in encoder prenet.
          encoder_normalize_before (bool): Whether to apply layernorm layer before
              encoder block.
          decoder_normalize_before (bool): Whether to apply layernorm layer before
              decoder block.
          encoder_concat_after (bool): Whether to concatenate attention layer's input
              and output in encoder.
          decoder_concat_after (bool): Whether to concatenate attention layer's input
              and output in decoder.
          positionwise_layer_type (str): Position-wise operation type.
          positionwise_conv_kernel_size (int): Kernel size in position wise conv 1d.
          reduction_factor (int): Reduction factor.
          spks (Optional[int]): Number of speakers. If set to > 1, assume that the
              sids will be provided as the input and use sid embedding layer.
          langs (Optional[int]): Number of languages. If set to > 1, assume that the
              lids will be provided as the input and use sid embedding layer.
          spk_embed_dim (Optional[int]): Speaker embedding dimension. If set to > 0,
              assume that spembs will be provided as the input.
          spk_embed_integration_type (str): How to integrate speaker embedding.
          use_gst (str): Whether to use global style token.
          gst_tokens (int): Number of GST embeddings.
          gst_heads (int): Number of heads in GST multihead attention.
          gst_conv_layers (int): Number of conv layers in GST.
          gst_conv_chans_list: (Sequence[int]): List of the number of channels of conv
              layers in GST.
          gst_conv_kernel_size (int): Kernel size of conv layers in GST.
          gst_conv_stride (int): Stride size of conv layers in GST.
          gst_gru_layers (int): Number of GRU layers in GST.
          gst_gru_units (int): Number of GRU units in GST.
          transformer_lr (float): Initial value of learning rate.
          transformer_warmup_steps (int): Optimizer warmup steps.
          transformer_enc_dropout_rate (float): Dropout rate in encoder except
              attention and positional encoding.
          transformer_enc_positional_dropout_rate (float): Dropout rate after encoder
              positional encoding.
          transformer_enc_attn_dropout_rate (float): Dropout rate in encoder
              self-attention module.
          transformer_dec_dropout_rate (float): Dropout rate in decoder except
              attention & positional encoding.
          transformer_dec_positional_dropout_rate (float): Dropout rate after decoder
              positional encoding.
          transformer_dec_attn_dropout_rate (float): Dropout rate in decoder
              self-attention module.
          transformer_enc_dec_attn_dropout_rate (float): Dropout rate in source
              attention module.
          init_type (str): How to initialize transformer parameters.
          init_enc_alpha (float): Initial value of alpha in scaled pos encoding of the
              encoder.
          init_dec_alpha (float): Initial value of alpha in scaled pos encoding of the
              decoder.
          eprenet_dropout_rate (float): Dropout rate in encoder prenet.
          dprenet_dropout_rate (float): Dropout rate in decoder prenet.
          postnet_dropout_rate (float): Dropout rate in postnet.
          use_masking (bool): Whether to apply masking for padded part in loss
              calculation.
          use_weighted_masking (bool): Whether to apply weighted masking in loss
              calculation.
          bce_pos_weight (float): Positive sample weight in bce calculation
              (only for use_masking=true).
          loss_type (str): How to calculate loss.
          use_guided_attn_loss (bool): Whether to use guided attention loss.
          num_heads_applied_guided_attn (int): Number of heads in each layer to apply
              guided attention loss.
          num_layers_applied_guided_attn (int): Number of layers to apply guided
              attention loss.
          modules_applied_guided_attn (Sequence[str]): List of module names to apply
              guided attention loss.
          guided_attn_loss_sigma (float) Sigma in guided attention loss.
          guided_attn_loss_lambda (float): Lambda in guided attention loss.
  
      """
      assert check_argument_types()
      super().__init__()
  
      # store hyperparameters
      self.idim = idim
      self.odim = odim
      self.eos = idim - 1
      self.reduction_factor = reduction_factor
      self.use_gst = use_gst
      self.use_guided_attn_loss = use_guided_attn_loss
      self.use_scaled_pos_enc = use_scaled_pos_enc
      self.loss_type = loss_type
      self.use_guided_attn_loss = use_guided_attn_loss
      if self.use_guided_attn_loss:
          if num_layers_applied_guided_attn == -1:
              self.num_layers_applied_guided_attn = elayers
          else:
              self.num_layers_applied_guided_attn = (
                  num_layers_applied_guided_attn
              )
          if num_heads_applied_guided_attn == -1:
              self.num_heads_applied_guided_attn = aheads
          else:
              self.num_heads_applied_guided_attn = (
                  num_heads_applied_guided_attn
              )
          self.modules_applied_guided_attn = modules_applied_guided_attn
  
      # use idx 0 as padding idx
      self.padding_idx = 0
  
      # get positional encoding class
      pos_enc_class = (
          ScaledPositionalEncoding
          if self.use_scaled_pos_enc
          else PositionalEncoding
      )
  
      # define transformer encoder
      if eprenet_conv_layers != 0:
          # encoder prenet
          encoder_input_layer = torch.nn.Sequential(
              EncoderPrenet(
                  idim=idim,
                  embed_dim=embed_dim,
                  elayers=0,
                  econv_layers=eprenet_conv_layers,
                  econv_chans=eprenet_conv_chans,
                  econv_filts=eprenet_conv_filts,
                  use_batch_norm=use_batch_norm,
                  dropout_rate=eprenet_dropout_rate,
                  padding_idx=self.padding_idx,
              ),
              torch.nn.Linear(eprenet_conv_chans, adim),
          )
      else:
          encoder_input_layer = torch.nn.Embedding(
              num_embeddings=idim,
              embedding_dim=adim,
              padding_idx=self.padding_idx,
          )
      self.encoder = Encoder(
          idim=idim,
          attention_dim=adim,
          attention_heads=aheads,
          linear_units=eunits,
          num_blocks=elayers,
          input_layer=encoder_input_layer,
          dropout_rate=transformer_enc_dropout_rate,
          positional_dropout_rate=transformer_enc_positional_dropout_rate,
          attention_dropout_rate=transformer_enc_attn_dropout_rate,
          pos_enc_class=pos_enc_class,
          normalize_before=encoder_normalize_before,
          concat_after=encoder_concat_after,
          positionwise_layer_type=positionwise_layer_type,
          positionwise_conv_kernel_size=positionwise_conv_kernel_size,
      )
  
      # define GST
      if self.use_gst:
          self.gst = StyleEncoder(
              idim=odim,  # the input is mel-spectrogram
              gst_tokens=gst_tokens,
              gst_token_dim=adim,
              gst_heads=gst_heads,
              conv_layers=gst_conv_layers,
              conv_chans_list=gst_conv_chans_list,
              conv_kernel_size=gst_conv_kernel_size,
              conv_stride=gst_conv_stride,
              gru_layers=gst_gru_layers,
              gru_units=gst_gru_units,
          )
  
      # define spk and lang embedding
      self.spks = None
      if spks is not None and spks > 1:
          self.spks = spks
          self.sid_emb = torch.nn.Embedding(spks, adim)
      self.langs = None
      if langs is not None and langs > 1:
          self.langs = langs
          self.lid_emb = torch.nn.Embedding(langs, adim)
  
      # define projection layer
      self.spk_embed_dim = None
      if spk_embed_dim is not None and spk_embed_dim > 0:
          self.spk_embed_dim = spk_embed_dim
          self.spk_embed_integration_type = spk_embed_integration_type
      if self.spk_embed_dim is not None:
          if self.spk_embed_integration_type == "add":
              self.projection = torch.nn.Linear(
                  self.spk_embed_dim, adim
              )
          else:
              self.projection = torch.nn.Linear(
                  adim + self.spk_embed_dim, adim
              )
  
      # define transformer decoder
      if dprenet_layers != 0:
          # decoder prenet
          decoder_input_layer = torch.nn.Sequential(
              DecoderPrenet(
                  idim=odim,
                  n_layers=dprenet_layers,
                  n_units=dprenet_units,
                  dropout_rate=dprenet_dropout_rate,
              ),
              torch.nn.Linear(dprenet_units, adim),
          )
      else:
          decoder_input_layer = "linear"
      self.decoder = Decoder(
          odim=odim,  # odim is needed when no prenet is used
          attention_dim=adim,
          attention_heads=aheads,
          linear_units=dunits,
          num_blocks=dlayers,
          dropout_rate=transformer_dec_dropout_rate,
          positional_dropout_rate=transformer_dec_positional_dropout_rate,
          self_attention_dropout_rate=transformer_dec_attn_dropout_rate,
          src_attention_dropout_rate=transformer_enc_dec_attn_dropout_rate,
          input_layer=decoder_input_layer,
          use_output_layer=False,
          pos_enc_class=pos_enc_class,
          normalize_before=decoder_normalize_before,
          concat_after=decoder_concat_after,
      )
  
      # define final projection
      self.feat_out = torch.nn.Linear(adim, odim * reduction_factor)
      self.prob_out = torch.nn.Linear(adim, reduction_factor)
  
      # define postnet
      self.postnet = (
          None
          if postnet_layers == 0
          else Postnet(
              idim=idim,
              odim=odim,
              n_layers=postnet_layers,
              n_chans=postnet_chans,
              n_filts=postnet_filts,
              use_batch_norm=use_batch_norm,
              dropout_rate=postnet_dropout_rate,
          )
      )
  
      # define loss function
      self.criterion = TransformerLoss(
          use_masking=use_masking,
          use_weighted_masking=use_weighted_masking,
          bce_pos_weight=bce_pos_weight,
      )
      if self.use_guided_attn_loss:
          self.attn_criterion = GuidedMultiHeadAttentionLoss(
              sigma=guided_attn_loss_sigma,
              alpha=guided_attn_loss_lambda,
          )
  
      # initialize parameters
      self._reset_parameters(
          init_type=init_type,
          init_enc_alpha=init_enc_alpha,
          init_dec_alpha=init_dec_alpha,
      )
  
imports: from typing import Dict, Optional, Sequence, Tuple
  import torch
  import torch.nn.functional
  from typeguard import check_argument_types
  from espnet2.torch_utils.device_funcs import force_gatherable
  from espnet2.torch_utils.initialize import initialize
  from espnet2.tts.abs_tts import AbsTTS
  from espnet2.tts.gst.style_encoder import StyleEncoder
  from espnet.nets.pytorch_backend.e2e_tts_transformer import GuidedMultiHeadAttentionLoss, TransformerLoss
  from espnet.nets.pytorch_backend.nets_utils import make_non_pad_mask, make_pad_mask
  from espnet.nets.pytorch_backend.tacotron2.decoder import Postnet
  from espnet.nets.pytorch_backend.tacotron2.decoder import Prenet
  from espnet.nets.pytorch_backend.tacotron2.encoder import Encoder
  from espnet.nets.pytorch_backend.transformer.attention import MultiHeadedAttention
  from espnet.nets.pytorch_backend.transformer.decoder import Decoder
  from espnet.nets.pytorch_backend.transformer.embedding import PositionalEncoding, ScaledPositionalEncoding
  from espnet.nets.pytorch_backend.transformer.encoder import Encoder
  from espnet.nets.pytorch_backend.transformer.mask import subsequent_mask
label_desc: Use image node_id 1 to create a new Transformer object from inherited base classes: AbsTTS with example: obj = Transformer(idim, odim, embed_dim, eprenet_conv_layers, eprenet_conv_chans, eprenet_conv_filts, dprenet_layers, dprenet_units, elayers, eunits, adim, aheads, dlayers, dunits, postnet_layers, postnet_chans, postnet_filts, positionwise_layer_type, positionwise_conv_kernel_size, use_scaled_pos_enc, use_batch_norm, encoder_normalize_before, decoder_normalize_before, encoder_concat_after, decoder_concat_after, reduction_factor, spks, langs, spk_embed_dim, spk_embed_integration_type, use_gst, gst_tokens, gst_heads, gst_conv_layers, gst_conv_chans_list, gst_conv_kernel_size, gst_conv_stride, gst_gru_layers, gst_gru_units, transformer_enc_dropout_rate, transformer_enc_positional_dropout_rate, transformer_enc_attn_dropout_rate, transformer_dec_dropout_rate, transformer_dec_positional_dropout_rate, transformer_dec_attn_dropout_rate, transformer_enc_dec_attn_dropout_rate, eprenet_dropout_rate, dprenet_dropout_rate, postnet_dropout_rate, init_type, init_enc_alpha, init_dec_alpha, use_masking, use_weighted_masking, bce_pos_weight, loss_type, use_guided_attn_loss, num_heads_applied_guided_attn, num_layers_applied_guided_attn, modules_applied_guided_attn, guided_attn_loss_sigma, guided_attn_loss_lambda)