class_docstr: A mixin class to support loading pre-trained checkpoints for modules
  that implement the BERT model.
  
  The BERT model was proposed in (`Devlin et al`. 2018)
  `BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding`_
  . A bidirectional Transformer language model pre-trained on large text
  corpora. Available model names include:
  
    * ``bert-base-uncased``: 12-layer, 768-hidden, 12-heads,
      110M parameters.
    * ``bert-large-uncased``: 24-layer, 1024-hidden, 16-heads,
      340M parameters.
    * ``bert-base-cased``: 12-layer, 768-hidden, 12-heads , 110M parameters.
    * ``bert-large-cased``: 24-layer, 1024-hidden, 16-heads,
      340M parameters.
    * ``bert-base-multilingual-uncased``: 102 languages, 12-layer,
      768-hidden, 12-heads, 110M parameters.
    * ``bert-base-multilingual-cased``: 104 languages, 12-layer, 768-hidden,
      12-heads, 110M parameters.
    * ``bert-base-chinese``: Chinese Simplified and Traditional, 12-layer,
      768-hidden, 12-heads, 110M parameters.
  
  We provide the following BERT classes:
  
    * :class:`~texar.tf.modules.BERTEncoder` for text encoding.
    * :class:`~texar.tf.modules.BERTClassifier` for text classification and
      sequence tagging.
  
  .. _`BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding`:
      https://arxiv.org/abs/1810.04805
class_docstr_tok: ["A","mixin","class","to","support","loading","pre-trained","checkpoints","for","modules","that","implement","the","BERT","model",".","The","BERT","model","was","proposed","in","(","`","Devlin","et","al","`",".","2018",")","`","BERT",":","Pre-training","of","Deep","Bidirectional","Transformers","for","Language","Understanding","`","_",".","A","bidirectional","Transformer","language","model","pre-trained","on","large","text","corpora",".","Available","model","names","include",":","*","``","bert-base-uncased","``",":","12-layer",",","768-hidden",",","12-heads",",","110M","parameters",".","*","``","bert-large-uncased","``",":","24-layer",",","1024-hidden",",","16-heads",",","340M","parameters",".","*","``","bert-base-cased","``",":","12-layer",",","768-hidden",",","12-heads",",","110M","parameters",".","*","``","bert-large-cased","``",":","24-layer",",","1024-hidden",",","16-heads",",","340M","parameters",".","*","``","bert-base-multilingual-uncased","``",":","102","languages",",","12-layer",",","768-hidden",",","12-heads",",","110M","parameters",".","*","``","bert-base-multilingual-cased","``",":","104","languages",",","12-layer",",","768-hidden",",","12-heads",",","110M","parameters",".","*","``","bert-base-chinese","``",":","Chinese","Simplified","and","Traditional",",","12-layer",",","768-hidden",",","12-heads",",","110M","parameters",".","We","provide","the","following","BERT","classes",":","*",":","class",":","`","~texar.tf.modules.BERTEncoder","`","for","text","encoding",".","*",":","class",":","`","~texar.tf.modules.BERTClassifier","`","for","text","classification","and","sequence","tagging",".","..","_","`","BERT",":","Pre-training","of","Deep","Bidirectional","Transformers","for","Language","Understanding","`",":","https",":","\/\/arxiv.org\/abs\/1810.04805"]
docstr: `https://github.com/google-research/bert/blob/master/modeling.py`
  
  Compute the union of the current variables and checkpoint variables.
  Because the variable scope of the original BERT and Texar
  implementation, we need to build a assignment map to match the
  variables.
docstr_tok: ["`","https",":","\/\/github.com\/google-research\/bert\/blob\/master\/modeling.py","`","Compute","the","union","of","the","current","variables","and","checkpoint","variables",".","Because","the","variable","scope","of","the","original","BERT","and","Texar","implementation",",","we","need","to","build","a","assignment","map","to","match","the","variables","."]
code: def _get_assignment_map_from_checkpoint(
      self, tvars, init_checkpoint, scope_name
  ):
      r"""`https://github.com/google-research/bert/blob/master/modeling.py`
  
      Compute the union of the current variables and checkpoint variables.
      Because the variable scope of the original BERT and Texar
      implementation, we need to build a assignment map to match the
      variables.
      """
      initialized_variable_names = {}
  
      name_to_variable = collections.OrderedDict()
      for var in tvars:
          name = var.name
          m = re.match("^(.*):\\d+$", name)
          if m is not None:
              name = m.group(1)
          name_to_variable[name] = var
  
      init_vars = tf.train.list_variables(init_checkpoint)
  
      assignment_map = {
          "bert/embeddings/word_embeddings": scope_name
          + "/word_embeddings/w",
          "bert/embeddings/token_type_embeddings": scope_name
          + "/token_type_embeddings/w",
          "bert/embeddings/position_embeddings": scope_name
          + "/position_embeddings/w",
          "bert/embeddings/LayerNorm/beta": scope_name
          + "/encoder/LayerNorm/beta",
          "bert/embeddings/LayerNorm/gamma": scope_name
          + "/encoder/LayerNorm/gamma",
      }
      for check_name, model_name in assignment_map.items():
          initialized_variable_names[model_name] = 1
          initialized_variable_names[model_name + ":0"] = 1
  
      for check_name, _ in init_vars:
          if check_name.startswith("bert"):
              if check_name.startswith("bert/embeddings"):
                  continue
              check_name_scope = check_name.replace(
                  "bert/", scope_name + "/"
              )
              model_name = re.sub(
                  "layer_\\d+/output/dense",
                  lambda x: x.group(0).replace(
                      "output/dense", "ffn/output"
                  ),
                  check_name_scope,
              )
              if model_name == check_name_scope:
                  model_name = re.sub(
                      "layer_\\d+/output/LayerNorm",
                      lambda x: x.group(0).replace(
                          "output/LayerNorm", "ffn/LayerNorm"
                      ),
                      check_name_scope,
                  )
              if model_name == check_name_scope:
                  model_name = re.sub(
                      "layer_\\d+/intermediate/dense",
                      lambda x: x.group(0).replace(
                          "intermediate/dense", "ffn/intermediate"
                      ),
                      check_name_scope,
                  )
              if model_name == check_name_scope:
                  model_name = re.sub(
                      "attention/output/dense",
                      "attention/self/output",
                      check_name_scope,
                  )
              if model_name == check_name_scope:
                  model_name = check_name_scope.replace(
                      "attention/output/LayerNorm", "output/LayerNorm"
                  )
  
              if model_name in name_to_variable.keys():
                  assignment_map[check_name] = model_name
                  initialized_variable_names[model_name] = 1
                  initialized_variable_names[model_name + ":0"] = 1
              else:
                  tf.logging.info(
                      "model name:{} not exist".format(model_name)
                  )
  
      return assignment_map, initialized_variable_names
  
file_path: texar/texar/tf/modules/pretrained/bert.py
imports: import collections
  import json
  import os
  import re
  from abc import ABCMeta
  import tensorflow
  from texar.tf.modules.pretrained.pretrained_base import PretrainedMixin
label_desc: Use image node_id 3 for calling the PretrainedBERTMixin obj's underlying member method code with example usage: obj._get_assignment_map_from_checkpoint(tvars, init_checkpoint, scope_name) and returns: assignment_map,  initialized_variable_names