class_docstr: Bert Model.
  
  Outputs: `Tuple` comprising various elements depending on the configuration
  (config) and inputs:
          **last_hidden_state**: ``torch.FloatTensor`` of shape ``(batch_size,
          sequence_length, hidden_size)``
                  Sequence of hidden-states at the output of the last layer of the
                  model.
          **pooler_output**: ``torch.FloatTensor`` of shape ``(batch_size,
          hidden_size)``
                  Last layer hidden-state of the first token of the sequence
                  (classification token)
                  further processed by a Linear layer and a Tanh activation function.
                  The Linear
                  layer weights are trained from the next sentence prediction
                  (classification)
                  objective during Bert pretraining. This output is usually *not* a
                  good summary
                  of the semantic content of the input, you're often better with
                  averaging or pooling
                  the sequence of hidden-states for the whole input sequence.
          **hidden_states**: (`optional`, returned when
          ``config.output_hidden_states=True``)
                  list of ``torch.FloatTensor`` (one for the output of each layer +
                  the output of the embeddings)
                  of shape ``(batch_size, sequence_length, hidden_size)``:
                  Hidden-states of the model at the output of each layer plus the
                  initial embedding outputs.
          **attentions**: (`optional`, returned when
          ``config.output_attentions=True``)
                  list of ``torch.FloatTensor`` (one for each layer) of shape
                  ``(batch_size, num_heads, sequence_length, sequence_length)``:
                  Attentions weights after the attention softmax, used to compute the
                  weighted average in the self-attention heads.
class_docstr_tok: ["Bert","Model",".","Outputs",":","`","Tuple","`","comprising","various","elements","depending","on","the","configuration","(","config",")","and","inputs",":","*","*","last_hidden_state","*","*",":","``","torch.FloatTensor","``","of","shape","``","(","batch_size",",","sequence_length",",","hidden_size",")","``","Sequence","of","hidden-states","at","the","output","of","the","last","layer","of","the","model",".","*","*","pooler_output","*","*",":","``","torch.FloatTensor","``","of","shape","``","(","batch_size",",","hidden_size",")","``","Last","layer","hidden-state","of","the","first","token","of","the","sequence","(","classification","token",")","further","processed","by","a","Linear","layer","and","a","Tanh","activation","function",".","The","Linear","layer","weights","are","trained","from","the","next","sentence","prediction","(","classification",")","objective","during","Bert","pretraining",".","This","output","is","usually","*","not","*","a","good","summary","of","the","semantic","content","of","the","input",",","you","'re","often","better","with","averaging","or","pooling","the","sequence","of","hidden-states","for","the","whole","input","sequence",".","*","*","hidden_states","*","*",":","(","`","optional","`",",","returned","when","``","config.output_hidden_states=True","``",")","list","of","``","torch.FloatTensor","``","(","one","for","the","output","of","each","layer","+","the","output","of","the","embeddings",")","of","shape","``","(","batch_size",",","sequence_length",",","hidden_size",")","``",":","Hidden-states","of","the","model","at","the","output","of","each","layer","plus","the","initial","embedding","outputs",".","*","*","attentions","*","*",":","(","`","optional","`",",","returned","when","``","config.output_attentions=True","``",")","list","of","``","torch.FloatTensor","``","(","one","for","each","layer",")","of","shape","``","(","batch_size",",","num_heads",",","sequence_length",",","sequence_length",")","``",":","Attentions","weights","after","the","attention","softmax",",","used","to","compute","the","weighted","average","in","the","self-attention","heads","."]
code: def forward(
      self,
      input_ids,
      attention_mask=None,
      token_type_ids=None,
      position_ids=None,
      features=None,
  ):
      if attention_mask is None:
          attention_mask = torch.ones_like(input_ids)
      if token_type_ids is None:
          token_type_ids = torch.zeros_like(input_ids)
  
      # We create a 3D attention mask from a 2D tensor mask.
      # Sizes are [batch_size, 1, 1, to_seq_length]
      # So we can broadcast to
      # [batch_size, num_heads, from_seq_length, to_seq_length]
      extended_attention_mask = attention_mask.unsqueeze(1).unsqueeze(2)
  
      # Since attention_mask is 1.0 for positions we want to attend and 0.0 for
      # masked positions, this operation will create a tensor which is 0.0 for
      # positions we want to attend and -10000.0 for masked positions.
      # Since we are adding it to the raw scores before the softmax, this is
      # effectively the same as removing these entirely.
      extended_attention_mask = extended_attention_mask.to(
          dtype=next(self.parameters()).dtype
      )  # fp16 compatibility
      extended_attention_mask = (
          1.0 - extended_attention_mask
      ) * -10000.0
  
      head_mask = [None] * self.config.num_hidden_layers
  
      embedding_output = self.embeddings(
          input_ids,
          position_ids=position_ids,
          token_type_ids=token_type_ids,
          features=features,
      )
      encoder_outputs = self.encoder(
          embedding_output, extended_attention_mask, head_mask=head_mask
      )
      sequence_output = encoder_outputs[0]
      pooled_output = self.pooler(sequence_output)
  
      outputs = (
          sequence_output,
          pooled_output,
      ) + encoder_outputs[
          1:
      ]  # add hidden_states and attentions if they are here
      # sequence_output, pooled_output, (hidden_states), (attentions)
      return outputs
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  from __future__ import unicode_literals
  import logging
  import math
  import torch
  from torch import nn
  from towhee.models.layers.activations import swish, gelu
label_desc: Use image node_id 3 for calling the BertMMT obj's underlying member method code with example usage: obj.forward(input_ids, attention_mask, token_type_ids, position_ids, features) and returns: outputs