class_docstr: RNN decoder with attention mechanism.
  
  Args:
      memory: The memory to query, e.g., the output of an RNN encoder. This
          tensor should be shaped `[batch_size, max_time, dim]`.
      memory_sequence_length (optional): A tensor of shape `[batch_size]`
          containing the sequence lengths for the batch
          entries in memory. If provided, the memory tensor rows are masked
          with zeros for values past the respective sequence lengths.
      cell (RNNCell, optional): An instance of `RNNCell`. If `None`, a cell
          is created as specified in :attr:`hparams`.
      cell_dropout_mode (optional): A Tensor taking value of
          :tf_main:`tf.estimator.ModeKeys <estimator/ModeKeys>`, which
          toggles dropout in the RNN cell (e.g., activates dropout in
          TRAIN mode). If `None`, :func:`~texar.tf.global_mode` is used.
          Ignored if :attr:`cell` is given.
      vocab_size (int, optional): Vocabulary size. Required if
          :attr:`output_layer` is `None`.
      output_layer (optional): An output layer that transforms cell output
          to logits. This can be:
  
          - A callable layer, e.g., an instance             of :tf_main:`tf.layers.Layer <layers/Layer>`.
          - A tensor. A dense layer will be created using the tensor             as the kernel weights. The bias of the dense layer is determined by            `hparams.output_layer_bias`. This can be used to tie the output             layer with the input embedding matrix, as proposed in             https://arxiv.org/pdf/1608.05859.pdf
          - `None`. A dense layer will be created based on attr:`vocab_size`            and `hparams.output_layer_bias`.
          - If no output layer after the cell output is needed, set             `(vocab_size=None, output_layer=tf.identity)`.
      cell_input_fn (callable, optional): A callable that produces RNN cell
          inputs. If `None` (default), the default is used:
          `lambda inputs, attention: tf.concat([inputs, attention], -1)`,
          which cancats regular RNN cell inputs with attentions.
      hparams (dict, optional): Hyperparameters. Missing
          hyperparamerter will be set to default values. See
          :meth:`default_hparams` for the hyperparameter sturcture and
          default values.
  
  See :meth:`~texar.tf.modules.RNNDecoderBase._build` for the inputs and outputs
  of the decoder. The decoder returns
  `(outputs, final_state, sequence_lengths)`, where `outputs` is an instance
  of :class:`~texar.tf.modules.AttentionRNNDecoderOutput`.
  
  Example:
  
      .. code-block:: python
  
          # Encodes the source
          enc_embedder = WordEmbedder(data.source_vocab.size, ...)
          encoder = UnidirectionalRNNEncoder(...)
  
          enc_outputs, _ = encoder(
              inputs=enc_embedder(data_batch['source_text_ids']),
              sequence_length=data_batch['source_length'])
  
          # Decodes while attending to the source
          dec_embedder = WordEmbedder(vocab_size=data.target_vocab.size, ...)
          decoder = AttentionRNNDecoder(
              memory=enc_outputs,
              memory_sequence_length=data_batch['source_length'],
              vocab_size=data.target_vocab.size)
  
          outputs, _, _ = decoder(
              decoding_strategy='train_greedy',
              inputs=dec_embedder(data_batch['target_text_ids']),
              sequence_length=data_batch['target_length']-1)
docstr: Returns a dictionary of hyperparameters with default values:
  
  Common hyperparameters are the same as in
  :class:`~texar.tf.modules.BasicRNNDecoder`.
  :meth:`~texar.tf.modules.BasicRNNDecoder.default_hparams`.
  Additional hyperparameters are for attention mechanism
  configuration.
  
  .. code-block:: python
  
      {
          "attention": {
              "type": "LuongAttention",
              "kwargs": {
                  "num_units": 256,
              },
              "attention_layer_size": None,
              "alignment_history": False,
              "output_attention": True,
          },
          # The following hyperparameters are the same as with
          # `BasicRNNDecoder`
          "rnn_cell": default_rnn_cell_hparams(),
          "max_decoding_length_train": None,
          "max_decoding_length_infer": None,
          "helper_train": {
              "type": "TrainingHelper",
              "kwargs": {}
          }
          "helper_infer": {
              "type": "SampleEmbeddingHelper",
              "kwargs": {}
          }
          "name": "attention_rnn_decoder"
      }
  
  Here:
  
  "attention": dict
      Attention hyperparameters, including:
  
      "type": str or class or instance
          The attention type. Can be an attention class, its name or
          module path, or a class instance. The class must be a subclass
          of :tf_main:`TF AttentionMechanism
          <contrib/seq2seq/AttentionMechanism>`. If class name is
          given, the class must be from modules
          :tf_main:`tf.contrib.seq2seq <contrib/seq2seq>` or
          :mod:`texar.tf.custom`.
  
          Example:
  
              .. code-block:: python
  
                  # class name
                  "type": "LuongAttention"
                  "type": "BahdanauAttention"
                  # module path
                  "type": "tf.contrib.seq2seq.BahdanauMonotonicAttention"
                  "type": "my_module.MyAttentionMechanismClass"
                  # class
                  "type": tf.contrib.seq2seq.LuongMonotonicAttention
                  # instance
                  "type": LuongAttention(...)
  
      "kwargs": dict
          keyword arguments for the attention class constructor.
          Arguments :attr:`memory` and
          :attr:`memory_sequence_length` should **not** be
          specified here because they are given to the decoder
          constructor. Ignored if "type" is an attention class
          instance. For example
  
          Example:
  
              .. code-block:: python
  
                  "type": "LuongAttention",
                  "kwargs": {
                      "num_units": 256,
                      "probability_fn": tf.nn.softmax
                  }
  
              Here "probability_fn" can also be set to the string name
              or module path to a probability function.
  
          "attention_layer_size": int or None
              The depth of the attention (output) layer. The context and
              cell output are fed into the attention layer to generate
              attention at each time step.
              If `None` (default), use the context as attention at each
              time step.
  
          "alignment_history": bool
              whether to store alignment history from all time steps
              in the final output state. (Stored as a time major
              `TensorArray` on which you must call `stack()`.)
  
          "output_attention": bool
              If `True` (default), the output at each time step is
              the attention value. This is the behavior of Luong-style
              attention mechanisms. If `False`, the output at each
              time step is the output of `cell`.  This is the
              beahvior of Bhadanau-style attention mechanisms.
              In both cases, the `attention` tensor is propagated to
              the next time step via the state and is used there.
              This flag only controls whether the attention mechanism
              is propagated up to the next cell in an RNN stack or to
              the top RNN output.
docstr_tok: ["Returns","a","dictionary","of","hyperparameters","with","default","values",":","Common","hyperparameters","are","the","same","as","in",":","class",":","`","~texar.tf.modules.BasicRNNDecoder","`",".",":","meth",":","`","~texar.tf.modules.BasicRNNDecoder.default_hparams","`",".","Additional","hyperparameters","are","for","attention","mechanism","configuration",".","..","code-block",":",":","python","{","``","attention","''",":","{","``","type","''",":","``","LuongAttention","''",",","``","kwargs","''",":","{","``","num_units","''",":","256",",","}",",","``","attention_layer_size","''",":","None",",","``","alignment_history","''",":","False",",","``","output_attention","''",":","True",",","}",",","#","The","following","hyperparameters","are","the","same","as","with","#","`","BasicRNNDecoder","`","``","rnn_cell","''",":","default_rnn_cell_hparams","(",")",",","``","max_decoding_length_train","''",":","None",",","``","max_decoding_length_infer","''",":","None",",","``","helper_train","''",":","{","``","type","''",":","``","TrainingHelper","''",",","``","kwargs","''",":","{","}","}","``","helper_infer","''",":","{","``","type","''",":","``","SampleEmbeddingHelper","''",",","``","kwargs","''",":","{","}","}","``","name","''",":","``","attention_rnn_decoder","''","}","Here",":","''","attention","''",":","dict","Attention","hyperparameters",",","including",":","``","type","''",":","str","or","class","or","instance","The","attention","type",".","Can","be","an","attention","class",",","its","name","or","module","path",",","or","a","class","instance",".","The","class","must","be","a","subclass","of",":","tf_main",":","`","TF","AttentionMechanism","<","contrib\/seq2seq\/AttentionMechanism",">","`",".","If","class","name","is","given",",","the","class","must","be","from","modules",":","tf_main",":","`","tf.contrib.seq2seq","<","contrib\/seq2seq",">","`","or",":","mod",":","`","texar.tf.custom","`",".","Example",":","..","code-block",":",":","python","#","class","name","``","type","''",":","``","LuongAttention","''","``","type","''",":","``","BahdanauAttention","''","#","module","path","``","type","''",":","``","tf.contrib.seq2seq.BahdanauMonotonicAttention","''","``","type","''",":","``","my_module.MyAttentionMechanismClass","''","#","class","``","type","''",":","tf.contrib.seq2seq.LuongMonotonicAttention","#","instance","``","type","''",":","LuongAttention","(","...",")","``","kwargs","''",":","dict","keyword","arguments","for","the","attention","class","constructor",".","Arguments",":","attr",":","`","memory","`","and",":","attr",":","`","memory_sequence_length","`","should","*","*","not","*","*","be","specified","here","because","they","are","given","to","the","decoder","constructor",".","Ignored","if","``","type","''","is","an","attention","class","instance",".","For","example","Example",":","..","code-block",":",":","python","``","type","''",":","``","LuongAttention","''",",","``","kwargs","''",":","{","``","num_units","''",":","256",",","``","probability_fn","''",":","tf.nn.softmax","}","Here","``","probability_fn","''","can","also","be","set","to","the","string","name","or","module","path","to","a","probability","function",".","``","attention_layer_size","''",":","int","or","None","The","depth","of","the","attention","(","output",")","layer",".","The","context","and","cell","output","are","fed","into","the","attention","layer","to","generate","attention","at","each","time","step",".","If","`","None","`","(","default",")",",","use","the","context","as","attention","at","each","time","step",".","``","alignment_history","''",":","bool","whether","to","store","alignment","history","from","all","time","steps","in","the","final","output","state",".","(","Stored","as","a","time","major","`","TensorArray","`","on","which","you","must","call","`","stack","(",")","`",".",")","``","output_attention","''",":","bool","If","`","True","`","(","default",")",",","the","output","at","each","time","step","is","the","attention","value",".","This","is","the","behavior","of","Luong-style","attention","mechanisms",".","If","`","False","`",",","the","output","at","each","time","step","is","the","output","of","`","cell","`",".","This","is","the","beahvior","of","Bhadanau-style","attention","mechanisms",".","In","both","cases",",","the","`","attention","`","tensor","is","propagated","to","the","next","time","step","via","the","state","and","is","used","there",".","This","flag","only","controls","whether","the","attention","mechanism","is","propagated","up","to","the","next","cell","in","an","RNN","stack","or","to","the","top","RNN","output","."]
code: def default_hparams():
      """Returns a dictionary of hyperparameters with default values:
  
      Common hyperparameters are the same as in
      :class:`~texar.tf.modules.BasicRNNDecoder`.
      :meth:`~texar.tf.modules.BasicRNNDecoder.default_hparams`.
      Additional hyperparameters are for attention mechanism
      configuration.
  
      .. code-block:: python
  
          {
              "attention": {
                  "type": "LuongAttention",
                  "kwargs": {
                      "num_units": 256,
                  },
                  "attention_layer_size": None,
                  "alignment_history": False,
                  "output_attention": True,
              },
              # The following hyperparameters are the same as with
              # `BasicRNNDecoder`
              "rnn_cell": default_rnn_cell_hparams(),
              "max_decoding_length_train": None,
              "max_decoding_length_infer": None,
              "helper_train": {
                  "type": "TrainingHelper",
                  "kwargs": {}
              }
              "helper_infer": {
                  "type": "SampleEmbeddingHelper",
                  "kwargs": {}
              }
              "name": "attention_rnn_decoder"
          }
  
      Here:
  
      "attention": dict
          Attention hyperparameters, including:
  
          "type": str or class or instance
              The attention type. Can be an attention class, its name or
              module path, or a class instance. The class must be a subclass
              of :tf_main:`TF AttentionMechanism
              <contrib/seq2seq/AttentionMechanism>`. If class name is
              given, the class must be from modules
              :tf_main:`tf.contrib.seq2seq <contrib/seq2seq>` or
              :mod:`texar.tf.custom`.
  
              Example:
  
                  .. code-block:: python
  
                      # class name
                      "type": "LuongAttention"
                      "type": "BahdanauAttention"
                      # module path
                      "type": "tf.contrib.seq2seq.BahdanauMonotonicAttention"
                      "type": "my_module.MyAttentionMechanismClass"
                      # class
                      "type": tf.contrib.seq2seq.LuongMonotonicAttention
                      # instance
                      "type": LuongAttention(...)
  
          "kwargs": dict
              keyword arguments for the attention class constructor.
              Arguments :attr:`memory` and
              :attr:`memory_sequence_length` should **not** be
              specified here because they are given to the decoder
              constructor. Ignored if "type" is an attention class
              instance. For example
  
              Example:
  
                  .. code-block:: python
  
                      "type": "LuongAttention",
                      "kwargs": {
                          "num_units": 256,
                          "probability_fn": tf.nn.softmax
                      }
  
                  Here "probability_fn" can also be set to the string name
                  or module path to a probability function.
  
              "attention_layer_size": int or None
                  The depth of the attention (output) layer. The context and
                  cell output are fed into the attention layer to generate
                  attention at each time step.
                  If `None` (default), use the context as attention at each
                  time step.
  
              "alignment_history": bool
                  whether to store alignment history from all time steps
                  in the final output state. (Stored as a time major
                  `TensorArray` on which you must call `stack()`.)
  
              "output_attention": bool
                  If `True` (default), the output at each time step is
                  the attention value. This is the behavior of Luong-style
                  attention mechanisms. If `False`, the output at each
                  time step is the output of `cell`.  This is the
                  beahvior of Bhadanau-style attention mechanisms.
                  In both cases, the `attention` tensor is propagated to
                  the next time step via the state and is used there.
                  This flag only controls whether the attention mechanism
                  is propagated up to the next cell in an RNN stack or to
                  the top RNN output.
      """
      hparams = RNNDecoderBase.default_hparams()
      hparams["name"] = "attention_rnn_decoder"
      hparams["attention"] = {
          "type": "LuongAttention",
          "kwargs": {
              "num_units": 256,
          },
          "attention_layer_size": None,
          "alignment_history": False,
          "output_attention": True,
      }
      return hparams
  
imports: import collections
  import copy
  import tensorflow
  from tensorflow.contrib.seq2seq import AttentionWrapper
  from tensorflow.python.util import nest
  from tensorflow.contrib.seq2seq import tile_batch
  from texar.tf.modules.decoders.rnn_decoder_base import RNNDecoderBase
  from texar.tf.utils import utils
label_desc: Use image node_id 2 for calling the AttentionRNNDecoder obj's underlying member method code with example usage: obj.default_hparams() and returns: hparams