class_docstr: Transformer encoder that applies multi-head self attention for encoding
  sequences.
  
  This module basically stacks
  :class:`~texar.tf.modules.encoders.MultiheadAttentionEncoder`,
  :class:`~texar.tf.modules.FeedForwardNetwork` and residual connections.
  
  This module supports two types of architectures, namely, the standard
  Transformer Encoder architecture first proposed in
  `(Vaswani et al.) "Attention is All You Need"`, and
  the variant first used in `(Devlin et al.)` BERT. See
  :meth:`default_hparams` for the nuance between the two types of
  architectures.
  
  Args:
      hparams (dict or HParams, optional): Hyperparameters. Missing
          hyperparameter will be set to default values. See
          :meth:`default_hparams` for the hyperparameter sturcture and
          default values.
  
  .. document private functions
  .. automethod:: _build
code: def __init__(self, hparams=None):
      EncoderBase.__init__(self, hparams)
  
      with tf.variable_scope(self.variable_scope):
          if self._hparams.initializer:
              tf.get_variable_scope().set_initializer(
                  layers.get_initializer(self._hparams.initializer)
              )
  
          self.multihead_attention_list = []
          self.poswise_networks = []
          for i in range(self._hparams.num_blocks):
              with tf.variable_scope("layer_{}".format(i)):
                  with tf.variable_scope("attention"):
                      mh_attn = MultiheadAttentionEncoder(
                          self._hparams.multihead_attention
                      )
                      self.multihead_attention_list.append(mh_attn)
  
                      if (
                          self._hparams.dim
                          != mh_attn.hparams.output_dim
                      ):
                          raise ValueError(
                              'The "dim" in the hparams of '
                              '"multihead_attention" should be equal to the '
                              '"dim" of TransformerEncoder'
                          )
  
                  pw_net = FeedForwardNetwork(
                      hparams=self._hparams["poswise_feedforward"]
                  )
                  final_dim = pw_net.hparams.layers[-1]["kwargs"][
                      "units"
                  ]
                  if self._hparams.dim != final_dim:
                      raise ValueError(
                          "The output dimenstion of "
                          '"poswise_feedforward" should be equal '
                          'to the "dim" of TransformerEncoder.'
                      )
                  self.poswise_networks.append(pw_net)
  
file_path: texar/texar/tf/modules/encoders/transformer_encoders.py
imports: import tensorflow
  from texar.tf.core import layers
  from texar.tf.utils import transformer_attentions, transformer_utils
  from texar.tf.modules.encoders.encoder_base import EncoderBase
  from texar.tf.modules.encoders.multihead_attention import MultiheadAttentionEncoder
  from texar.tf.modules.networks.networks import FeedForwardNetwork
  from texar.tf.utils.shapes import shape_list
  from texar.tf.utils.mode import is_train_mode
label_desc: Use image node_id 1 to create a new TransformerEncoder object from inherited base classes: EncoderBase with example: obj = TransformerEncoder(hparams)