class_docstr: A module that directly utilizes the pretrained models in `transformers`_ to produce BERT representations.
  
  While mainly tailored to provide input preparation and post-processing for the BERT model,
  it is also compatiable with other pretrained language models like XLNet, RoBERTa and ELECTRA, etc.
  
  Args:
      model (str):
          Path or name of the pretrained models registered in `transformers`_, e.g., ``'bert-base-cased'``.
      n_layers (int):
          The number of layers from the model to use.
          If 0, uses all layers.
      n_out (int):
          The requested size of the embeddings.
          If 0, uses the size of the pretrained embedding model.
      pad_index (int):
          The index of the padding token in the BERT vocabulary. Default: 0.
      max_len (int):
          Sequences should not exceed the specfied max length. Default: 512.
      dropout (float):
          The dropout ratio of BERT layers. Default: 0.
          This value will be passed into the :class:`ScalarMix` layer.
      requires_grad (bool):
          If ``True``, the model parameters will be updated together with the downstream task.
          Default: ``False``.
  
  .. _transformers:
      https://github.com/huggingface/transformers
code: def __init__(
      self,
      model,
      n_layers,
      n_out,
      pad_index=0,
      max_len=512,
      dropout=0,
      requires_grad=False,
  ):
      super().__init__()
  
      self.model = model
      self.bert = AutoModel.from_pretrained(
          model,
          config=AutoConfig.from_pretrained(
              model, output_hidden_states=True
          ),
      )
      self.bert = self.bert.requires_grad_(requires_grad)
      self.n_layers = n_layers or self.bert.config.num_hidden_layers
      self.hidden_size = self.bert.config.hidden_size
      self.n_out = n_out or self.hidden_size
      self.pad_index = pad_index
      self.max_len = max_len
      self.dropout = dropout
      self.requires_grad = requires_grad
  
      self.scalar_mix = ScalarMix(self.n_layers, dropout)
      self.projection = (
          nn.Linear(self.hidden_size, self.n_out, False)
          if self.hidden_size != n_out
          else nn.Identity()
      )
  
file_path: underthesea/underthesea/modules/bert.py
imports: import torch.nn
  from torch.nn.utils.rnn import pad_sequence
  from transformers import AutoConfig, AutoModel
  from underthesea.modules.scalar_mix import ScalarMix
label_desc: Use image node_id 1 to create a new BertEmbedding object from inherited base classes: nn with example: obj = BertEmbedding(model, n_layers, n_out, pad_index, max_len, dropout, requires_grad)