class_docstr: Raw XLNet module for encoding sequences. Please see
  :class:`~texar.tf.modules.PretrainedXLNetMixin` for a brief description
  of XLNet.
  
  Args:
      pretrained_model_name (optional): a `str`, the name
          of pre-trained model (e.g., ``xlnet-based-cased``). Please refer to
          :class:`~texar.tf.modules.PretrainedXLNetMixin` for
          all supported models.
          If `None`, the model name in :attr:`hparams` is used.
      cache_dir (optional): the path to a folder in which the
          pre-trained models will be cached. If `None` (default),
          a default directory (``texar_data`` folder under user's home
          directory) will be used.
      hparams (dict or HParams, optional): Hyperparameters. Missing
          hyperparameter will be set to default values. See
          :meth:`default_hparams` for the hyperparameter structure
          and default values.
  
  .. document private functions
  .. automethod:: _build
class_docstr_tok: ["Raw","XLNet","module","for","encoding","sequences",".","Please","see",":","class",":","`","~texar.tf.modules.PretrainedXLNetMixin","`","for","a","brief","description","of","XLNet",".","Args",":","pretrained_model_name","(","optional",")",":","a","`","str","`",",","the","name","of","pre-trained","model","(","e.g.",",","``","xlnet-based-cased","``",")",".","Please","refer","to",":","class",":","`","~texar.tf.modules.PretrainedXLNetMixin","`","for","all","supported","models",".","If","`","None","`",",","the","model","name","in",":","attr",":","`","hparams","`","is","used",".","cache_dir","(","optional",")",":","the","path","to","a","folder","in","which","the","pre-trained","models","will","be","cached",".","If","`","None","`","(","default",")",",","a","default","directory","(","``","texar_data","``","folder","under","user","'s","home","directory",")","will","be","used",".","hparams","(","dict","or","HParams",",","optional",")",":","Hyperparameters",".","Missing","hyperparameter","will","be","set","to","default","values",".","See",":","meth",":","`","default_hparams","`","for","the","hyperparameter","structure","and","default","values",".","..","document","private","functions","..","automethod",":",":","_build"]
docstr: Compute XLNet representations for the input.
  
  Args:
      token_ids: Shape `[batch_size, max_time]`.
      segment_ids: Shape `[batch_size, max_time]`.
      input_mask: Float tensor of shape `[batch_size, max_time]`. Note that
          positions with value 1 are masked out.
      memory: Memory from previous batches. A list of length `num_layers`,
          each tensor of shape `[batch_size, mem_len, hidden_dim]`.
      permute_mask: The permutation mask. Float tensor of shape
          `[batch_size, max_time, max_time]`.
          A value of 0 for ``permute_mask[i, j, k]`` indicates that
          position `i` attends to position `j` in batch `k`.
      target_mapping: The target token mapping. Float tensor of shape
          `[batch_size, num_targets, max_time]`.
          A value of 1 for ``target_mapping[i, j, k]`` indicates that
          the `i`-th target token (in order of permutation) in batch `k`
          is the token at position `j`.
          Each row ``target_mapping[i, :, k]`` can have no more than one
          value of 1.
      bi_data (bool): Whether to use bidirectional data input pipeline.
      clamp_len (int): Clamp all relative distances larger than
          :attr:`clamp_len`. A value of -1 means no clamping.
      cache_len (int): Length of memory (number of tokens) to cache.
      same_length (bool): Whether to use the same attention length for
          each token.
      attn_type (str): Attention type. Supported values are `"uni"`
          and `"bi"`.
      two_stream (bool): Whether to use two-stream attention. Only set to
          `True` when pre-training or generating text. Defaults to
          `False`.
  
  Returns: A tuple of `(output, new_memory)`:
  
      - **output**: The final layer output representations. Shape
        `[batch_size, max_time, hidden_dim]`.
      - **new_memory**: The memory of the current batch.
        If `cache_len` is 0, then `new_memory` is `None`. Otherwise, it is
        a list of length `num_layers`, each tensor of shape
        `[batch_size, cache_len, hidden_dim]`.
        This can be used as the :attr:`memory` argument in the next batch.
docstr_tok: ["Compute","XLNet","representations","for","the","input",".","Args",":","token_ids",":","Shape","`","[","batch_size",",","max_time","]","`",".","segment_ids",":","Shape","`","[","batch_size",",","max_time","]","`",".","input_mask",":","Float","tensor","of","shape","`","[","batch_size",",","max_time","]","`",".","Note","that","positions","with","value","1","are","masked","out",".","memory",":","Memory","from","previous","batches",".","A","list","of","length","`","num_layers","`",",","each","tensor","of","shape","`","[","batch_size",",","mem_len",",","hidden_dim","]","`",".","permute_mask",":","The","permutation","mask",".","Float","tensor","of","shape","`","[","batch_size",",","max_time",",","max_time","]","`",".","A","value","of","0","for","``","permute_mask","[","i",",","j",",","k","]","``","indicates","that","position","`","i","`","attends","to","position","`","j","`","in","batch","`","k","`",".","target_mapping",":","The","target","token","mapping",".","Float","tensor","of","shape","`","[","batch_size",",","num_targets",",","max_time","]","`",".","A","value","of","1","for","``","target_mapping","[","i",",","j",",","k","]","``","indicates","that","the","`","i","`","-th","target","token","(","in","order","of","permutation",")","in","batch","`","k","`","is","the","token","at","position","`","j","`",".","Each","row","``","target_mapping","[","i",",",":",",","k","]","``","can","have","no","more","than","one","value","of","1.","bi_data","(","bool",")",":","Whether","to","use","bidirectional","data","input","pipeline",".","clamp_len","(","int",")",":","Clamp","all","relative","distances","larger","than",":","attr",":","`","clamp_len","`",".","A","value","of","-1","means","no","clamping",".","cache_len","(","int",")",":","Length","of","memory","(","number","of","tokens",")","to","cache",".","same_length","(","bool",")",":","Whether","to","use","the","same","attention","length","for","each","token",".","attn_type","(","str",")",":","Attention","type",".","Supported","values","are","`","``","uni","''","`","and","`","``","bi","''","`",".","two_stream","(","bool",")",":","Whether","to","use","two-stream","attention",".","Only","set","to","`","True","`","when","pre-training","or","generating","text",".","Defaults","to","`","False","`",".","Returns",":","A","tuple","of","`","(","output",",","new_memory",")","`",":","-","*","*","output","*","*",":","The","final","layer","output","representations",".","Shape","`","[","batch_size",",","max_time",",","hidden_dim","]","`",".","-","*","*","new_memory","*","*",":","The","memory","of","the","current","batch",".","If","`","cache_len","`","is","0",",","then","`","new_memory","`","is","`","None","`",".","Otherwise",",","it","is","a","list","of","length","`","num_layers","`",",","each","tensor","of","shape","`","[","batch_size",",","cache_len",",","hidden_dim","]","`",".","This","can","be","used","as","the",":","attr",":","`","memory","`","argument","in","the","next","batch","."]
code: def _build(
      self,
      token_ids,
      segment_ids=None,
      input_mask=None,
      memory=None,
      permute_mask=None,
      target_mapping=None,
      bi_data=False,
      clamp_len=None,
      cache_len=0,
      same_length=False,
      attn_type="bi",
      two_stream=False,
      mode=None,
  ):
      r"""Compute XLNet representations for the input.
  
      Args:
          token_ids: Shape `[batch_size, max_time]`.
          segment_ids: Shape `[batch_size, max_time]`.
          input_mask: Float tensor of shape `[batch_size, max_time]`. Note that
              positions with value 1 are masked out.
          memory: Memory from previous batches. A list of length `num_layers`,
              each tensor of shape `[batch_size, mem_len, hidden_dim]`.
          permute_mask: The permutation mask. Float tensor of shape
              `[batch_size, max_time, max_time]`.
              A value of 0 for ``permute_mask[i, j, k]`` indicates that
              position `i` attends to position `j` in batch `k`.
          target_mapping: The target token mapping. Float tensor of shape
              `[batch_size, num_targets, max_time]`.
              A value of 1 for ``target_mapping[i, j, k]`` indicates that
              the `i`-th target token (in order of permutation) in batch `k`
              is the token at position `j`.
              Each row ``target_mapping[i, :, k]`` can have no more than one
              value of 1.
          bi_data (bool): Whether to use bidirectional data input pipeline.
          clamp_len (int): Clamp all relative distances larger than
              :attr:`clamp_len`. A value of -1 means no clamping.
          cache_len (int): Length of memory (number of tokens) to cache.
          same_length (bool): Whether to use the same attention length for
              each token.
          attn_type (str): Attention type. Supported values are `"uni"`
              and `"bi"`.
          two_stream (bool): Whether to use two-stream attention. Only set to
              `True` when pre-training or generating text. Defaults to
              `False`.
  
      Returns: A tuple of `(output, new_memory)`:
  
          - **output**: The final layer output representations. Shape
            `[batch_size, max_time, hidden_dim]`.
          - **new_memory**: The memory of the current batch.
            If `cache_len` is 0, then `new_memory` is `None`. Otherwise, it is
            a list of length `num_layers`, each tensor of shape
            `[batch_size, cache_len, hidden_dim]`.
            This can be used as the :attr:`memory` argument in the next batch.
      """
      return self._execute(
          self.word_embedder(token_ids),
          segment_ids=segment_ids,
          input_mask=input_mask,
          memory=memory,
          permute_mask=permute_mask,
          target_mapping=target_mapping,
          bi_data=bi_data,
          clamp_len=clamp_len,
          cache_len=cache_len,
          same_length=same_length,
          attn_type=attn_type,
          two_stream=two_stream,
          mode=mode,
      )
  
imports: import tensorflow
  from texar.tf.utils.mode import is_train_mode
  from texar.tf.core.layers import get_initializer, get_layer
  from texar.tf.modules.embedders.embedders import WordEmbedder
  from texar.tf.modules.encoders.encoder_base import EncoderBase
  from texar.tf.modules.pretrained.xlnet import PretrainedXLNetMixin
  from texar.tf.modules.pretrained.xlnet_utils import PositionWiseFF, RelativePositionalEncoding, RelativeMutiheadAttention
  from texar.tf.utils.utils import dict_fetch
label_desc: Use image node_id 8 for calling the XLNetEncoder obj's underlying member method code with example usage: obj._build(token_ids, segment_ids, input_mask, memory, permute_mask, target_mapping, bi_data, clamp_len, cache_len, same_length, attn_type, two_stream, mode) and returns: self