class_docstr: Bert Model.
  
  Outputs: `Tuple` comprising various elements depending on the configuration
  (config) and inputs:
          **last_hidden_state**: ``torch.FloatTensor`` of shape ``(batch_size,
          sequence_length, hidden_size)``
                  Sequence of hidden-states at the output of the last layer of the
                  model.
          **pooler_output**: ``torch.FloatTensor`` of shape ``(batch_size,
          hidden_size)``
                  Last layer hidden-state of the first token of the sequence
                  (classification token)
                  further processed by a Linear layer and a Tanh activation function.
                  The Linear
                  layer weights are trained from the next sentence prediction
                  (classification)
                  objective during Bert pretraining. This output is usually *not* a
                  good summary
                  of the semantic content of the input, you're often better with
                  averaging or pooling
                  the sequence of hidden-states for the whole input sequence.
          **hidden_states**: (`optional`, returned when
          ``config.output_hidden_states=True``)
                  list of ``torch.FloatTensor`` (one for the output of each layer +
                  the output of the embeddings)
                  of shape ``(batch_size, sequence_length, hidden_size)``:
                  Hidden-states of the model at the output of each layer plus the
                  initial embedding outputs.
          **attentions**: (`optional`, returned when
          ``config.output_attentions=True``)
                  list of ``torch.FloatTensor`` (one for each layer) of shape
                  ``(batch_size, num_heads, sequence_length, sequence_length)``:
                  Attentions weights after the attention softmax, used to compute the
                  weighted average in the self-attention heads.
class_docstr_tok: ["Bert","Model",".","Outputs",":","`","Tuple","`","comprising","various","elements","depending","on","the","configuration","(","config",")","and","inputs",":","*","*","last_hidden_state","*","*",":","``","torch.FloatTensor","``","of","shape","``","(","batch_size",",","sequence_length",",","hidden_size",")","``","Sequence","of","hidden-states","at","the","output","of","the","last","layer","of","the","model",".","*","*","pooler_output","*","*",":","``","torch.FloatTensor","``","of","shape","``","(","batch_size",",","hidden_size",")","``","Last","layer","hidden-state","of","the","first","token","of","the","sequence","(","classification","token",")","further","processed","by","a","Linear","layer","and","a","Tanh","activation","function",".","The","Linear","layer","weights","are","trained","from","the","next","sentence","prediction","(","classification",")","objective","during","Bert","pretraining",".","This","output","is","usually","*","not","*","a","good","summary","of","the","semantic","content","of","the","input",",","you","'re","often","better","with","averaging","or","pooling","the","sequence","of","hidden-states","for","the","whole","input","sequence",".","*","*","hidden_states","*","*",":","(","`","optional","`",",","returned","when","``","config.output_hidden_states=True","``",")","list","of","``","torch.FloatTensor","``","(","one","for","the","output","of","each","layer","+","the","output","of","the","embeddings",")","of","shape","``","(","batch_size",",","sequence_length",",","hidden_size",")","``",":","Hidden-states","of","the","model","at","the","output","of","each","layer","plus","the","initial","embedding","outputs",".","*","*","attentions","*","*",":","(","`","optional","`",",","returned","when","``","config.output_attentions=True","``",")","list","of","``","torch.FloatTensor","``","(","one","for","each","layer",")","of","shape","``","(","batch_size",",","num_heads",",","sequence_length",",","sequence_length",")","``",":","Attentions","weights","after","the","attention","softmax",",","used","to","compute","the","weighted","average","in","the","self-attention","heads","."]
code: def __init__(self, config):
      super().__init__()
      self.config = config
      self.embeddings = BertEmbeddings(config)
      self.encoder = BertEncoder(config)
      self.pooler = BertPooler(config)
  
      # Weights initialization
      self.apply(self._init_weights)
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  from __future__ import unicode_literals
  import logging
  import math
  import torch
  from torch import nn
  from towhee.models.layers.activations import swish, gelu
label_desc: Use image node_id 1 to create a new BertMMT object from inherited base classes: nn with example: obj = BertMMT(config)