class_docstr: Raw XLNet module for encoding sequences. Please see
  :class:`~texar.tf.modules.PretrainedXLNetMixin` for a brief description
  of XLNet.
  
  Args:
      pretrained_model_name (optional): a `str`, the name
          of pre-trained model (e.g., ``xlnet-based-cased``). Please refer to
          :class:`~texar.tf.modules.PretrainedXLNetMixin` for
          all supported models.
          If `None`, the model name in :attr:`hparams` is used.
      cache_dir (optional): the path to a folder in which the
          pre-trained models will be cached. If `None` (default),
          a default directory (``texar_data`` folder under user's home
          directory) will be used.
      hparams (dict or HParams, optional): Hyperparameters. Missing
          hyperparameter will be set to default values. See
          :meth:`default_hparams` for the hyperparameter structure
          and default values.
  
  .. document private functions
  .. automethod:: _build
class_docstr_tok: ["Raw","XLNet","module","for","encoding","sequences",".","Please","see",":","class",":","`","~texar.tf.modules.PretrainedXLNetMixin","`","for","a","brief","description","of","XLNet",".","Args",":","pretrained_model_name","(","optional",")",":","a","`","str","`",",","the","name","of","pre-trained","model","(","e.g.",",","``","xlnet-based-cased","``",")",".","Please","refer","to",":","class",":","`","~texar.tf.modules.PretrainedXLNetMixin","`","for","all","supported","models",".","If","`","None","`",",","the","model","name","in",":","attr",":","`","hparams","`","is","used",".","cache_dir","(","optional",")",":","the","path","to","a","folder","in","which","the","pre-trained","models","will","be","cached",".","If","`","None","`","(","default",")",",","a","default","directory","(","``","texar_data","``","folder","under","user","'s","home","directory",")","will","be","used",".","hparams","(","dict","or","HParams",",","optional",")",":","Hyperparameters",".","Missing","hyperparameter","will","be","set","to","default","values",".","See",":","meth",":","`","default_hparams","`","for","the","hyperparameter","structure","and","default","values",".","..","document","private","functions","..","automethod",":",":","_build"]
docstr: Returns a dictionary of hyperparameters with default values.
  
  * The encoder arch is determined by the constructor argument
    :attr:`pretrained_model_name` if it's specified. In this case,
    `hparams` are ignored.
  * Otherwise, the encoder arch is determined by
    `hparams['pretrained_model_name']` if it's specified. All other
    configurations in `hparams` are ignored.
  * If the above two are `None`, the encoder arch is defined by
    the configurations in `hparams` and weights are randomly initialized.
  
  .. code-block:: python
  
      {
          "name": "xlnet_encoder",
          "pretrained_model_name": "xlnet-base-cased",
          "untie_r": True,
          "num_layers": 12,
          "mem_len": 0,
          "reuse_len": 0,
          "initializer": None,
          "num_heads": 12,
          "hidden_dim": 768,
          "head_dim": 64,
          "dropout": 0.1,
          "attention_dropout": 0.1,
          "use_segments": True,
          "ffn_inner_dim": 3072,
          "activation": 'gelu',
          "vocab_size": 32000,
          "max_seq_len": 512,
      }
  
  
  
  Here:
  
  The default parameters are values for cased XLNet-Base model.
  
  
  `"pretrained_model_name"`: str or None
       The name of the pre-trained bert model. If None, the model
       will be randomly initialized.
  
  `"untie_r"`: bool
      Boolean value to indicate if biases should be untied for all the
      layers
  
  `"num_layers"`: int
      Number of layers in the network
  
  `"mem_len"`: int
      Length of the memory to be used during attention score calculation.
  
  `"reuse_len"`: int
      Length of the memory that can be re-used
  
  `"initializer"`: dict, optional
      Hyperparameters of the default initializer that initializes
      variables created in this module.
      See :func:`~texar.tf.core.get_initializer` for details.
  
  `"num_heads"`: int
      Number of heads in the attention
  
  `"hidden_dim"`: int
      Hidden dimension of the embeddings
  
  `"head_dim"`: int
      Size of the vectors after head projection.
  
  `"dropout"`: float
      Dropout rate for layers
  
  `"attention_dropout"`: float
      Dropout rate for attention layers
  
  `"use_segments"`: bool
      Boolean to indicate if the input has segments
  
  `"ffn_inner_dim"`: int
      Dimension of PositionWise FF network's hidden layer
  
  `"activation"`: str or callable
      Activation function applied to the output of the PositionWise FF.
      See :func:`~texar.tf.core.get_activation_fn` for more details.
  
  `"vocab_size"`: int
      The vocabulary size of `inputs` in `XLNet`.
  
  `"max_seq_len"`: int
      Maximum len of the sequence allowed in one segment
  
  `"name"`: str
      Name of the module.
docstr_tok: ["Returns","a","dictionary","of","hyperparameters","with","default","values",".","*","The","encoder","arch","is","determined","by","the","constructor","argument",":","attr",":","`","pretrained_model_name","`","if","it","'s","specified",".","In","this","case",",","`","hparams","`","are","ignored",".","*","Otherwise",",","the","encoder","arch","is","determined","by","`","hparams","[","'pretrained_model_name","'","]","`","if","it","'s","specified",".","All","other","configurations","in","`","hparams","`","are","ignored",".","*","If","the","above","two","are","`","None","`",",","the","encoder","arch","is","defined","by","the","configurations","in","`","hparams","`","and","weights","are","randomly","initialized",".","..","code-block",":",":","python","{","``","name","''",":","``","xlnet_encoder","''",",","``","pretrained_model_name","''",":","``","xlnet-base-cased","''",",","``","untie_r","''",":","True",",","``","num_layers","''",":","12",",","``","mem_len","''",":","0",",","``","reuse_len","''",":","0",",","``","initializer","''",":","None",",","``","num_heads","''",":","12",",","``","hidden_dim","''",":","768",",","``","head_dim","''",":","64",",","``","dropout","''",":","0.1",",","``","attention_dropout","''",":","0.1",",","``","use_segments","''",":","True",",","``","ffn_inner_dim","''",":","3072",",","``","activation","''",":","'gelu","'",",","``","vocab_size","''",":","32000",",","``","max_seq_len","''",":","512",",","}","Here",":","The","default","parameters","are","values","for","cased","XLNet-Base","model",".","`","``","pretrained_model_name","''","`",":","str","or","None","The","name","of","the","pre-trained","bert","model",".","If","None",",","the","model","will","be","randomly","initialized",".","`","``","untie_r","''","`",":","bool","Boolean","value","to","indicate","if","biases","should","be","untied","for","all","the","layers","`","``","num_layers","''","`",":","int","Number","of","layers","in","the","network","`","``","mem_len","''","`",":","int","Length","of","the","memory","to","be","used","during","attention","score","calculation",".","`","``","reuse_len","''","`",":","int","Length","of","the","memory","that","can","be","re-used","`","``","initializer","''","`",":","dict",",","optional","Hyperparameters","of","the","default","initializer","that","initializes","variables","created","in","this","module",".","See",":","func",":","`","~texar.tf.core.get_initializer","`","for","details",".","`","``","num_heads","''","`",":","int","Number","of","heads","in","the","attention","`","``","hidden_dim","''","`",":","int","Hidden","dimension","of","the","embeddings","`","``","head_dim","''","`",":","int","Size","of","the","vectors","after","head","projection",".","`","``","dropout","''","`",":","float","Dropout","rate","for","layers","`","``","attention_dropout","''","`",":","float","Dropout","rate","for","attention","layers","`","``","use_segments","''","`",":","bool","Boolean","to","indicate","if","the","input","has","segments","`","``","ffn_inner_dim","''","`",":","int","Dimension","of","PositionWise","FF","network","'s","hidden","layer","`","``","activation","''","`",":","str","or","callable","Activation","function","applied","to","the","output","of","the","PositionWise","FF",".","See",":","func",":","`","~texar.tf.core.get_activation_fn","`","for","more","details",".","`","``","vocab_size","''","`",":","int","The","vocabulary","size","of","`","inputs","`","in","`","XLNet","`",".","`","``","max_seq_len","''","`",":","int","Maximum","len","of","the","sequence","allowed","in","one","segment","`","``","name","''","`",":","str","Name","of","the","module","."]
code: def default_hparams():
      r"""Returns a dictionary of hyperparameters with default values.
  
      * The encoder arch is determined by the constructor argument
        :attr:`pretrained_model_name` if it's specified. In this case,
        `hparams` are ignored.
      * Otherwise, the encoder arch is determined by
        `hparams['pretrained_model_name']` if it's specified. All other
        configurations in `hparams` are ignored.
      * If the above two are `None`, the encoder arch is defined by
        the configurations in `hparams` and weights are randomly initialized.
  
      .. code-block:: python
  
          {
              "name": "xlnet_encoder",
              "pretrained_model_name": "xlnet-base-cased",
              "untie_r": True,
              "num_layers": 12,
              "mem_len": 0,
              "reuse_len": 0,
              "initializer": None,
              "num_heads": 12,
              "hidden_dim": 768,
              "head_dim": 64,
              "dropout": 0.1,
              "attention_dropout": 0.1,
              "use_segments": True,
              "ffn_inner_dim": 3072,
              "activation": 'gelu',
              "vocab_size": 32000,
              "max_seq_len": 512,
          }
  
  
  
      Here:
  
      The default parameters are values for cased XLNet-Base model.
  
  
      `"pretrained_model_name"`: str or None
           The name of the pre-trained bert model. If None, the model
           will be randomly initialized.
  
      `"untie_r"`: bool
          Boolean value to indicate if biases should be untied for all the
          layers
  
      `"num_layers"`: int
          Number of layers in the network
  
      `"mem_len"`: int
          Length of the memory to be used during attention score calculation.
  
      `"reuse_len"`: int
          Length of the memory that can be re-used
  
      `"initializer"`: dict, optional
          Hyperparameters of the default initializer that initializes
          variables created in this module.
          See :func:`~texar.tf.core.get_initializer` for details.
  
      `"num_heads"`: int
          Number of heads in the attention
  
      `"hidden_dim"`: int
          Hidden dimension of the embeddings
  
      `"head_dim"`: int
          Size of the vectors after head projection.
  
      `"dropout"`: float
          Dropout rate for layers
  
      `"attention_dropout"`: float
          Dropout rate for attention layers
  
      `"use_segments"`: bool
          Boolean to indicate if the input has segments
  
      `"ffn_inner_dim"`: int
          Dimension of PositionWise FF network's hidden layer
  
      `"activation"`: str or callable
          Activation function applied to the output of the PositionWise FF.
          See :func:`~texar.tf.core.get_activation_fn` for more details.
  
      `"vocab_size"`: int
          The vocabulary size of `inputs` in `XLNet`.
  
      `"max_seq_len"`: int
          Maximum len of the sequence allowed in one segment
  
      `"name"`: str
          Name of the module.
      """
  
      return {
          "name": "xlnet_encoder",
          "pretrained_model_name": "xlnet-base-cased",
          "untie_r": True,
          "num_layers": 12,
          "mem_len": 0,
          "reuse_len": 0,
          # initializer
          "initializer": None,
          # layer
          "num_heads": 12,
          "hidden_dim": 768,
          "head_dim": 64,
          "dropout": 0.1,
          "attention_dropout": 0.1,
          "use_segments": True,
          # ffn
          "ffn_inner_dim": 3072,
          "activation": "gelu",
          # embedding
          "vocab_size": 32000,
          "max_seq_len": 512,
          "@no_typecheck": ["pretrained_model_name"],
      }
  
imports: import tensorflow
  from texar.tf.utils.mode import is_train_mode
  from texar.tf.core.layers import get_initializer, get_layer
  from texar.tf.modules.embedders.embedders import WordEmbedder
  from texar.tf.modules.encoders.encoder_base import EncoderBase
  from texar.tf.modules.pretrained.xlnet import PretrainedXLNetMixin
  from texar.tf.modules.pretrained.xlnet_utils import PositionWiseFF, RelativePositionalEncoding, RelativeMutiheadAttention
  from texar.tf.utils.utils import dict_fetch
label_desc: Use image node_id 3 for calling the XLNetEncoder obj's underlying member method code with example usage: obj.default_hparams() and returns: dict