title: Python: BERT Error - Some weights of the model checkpoint at were not used when initializing BertModel
body: <p>I am creating an entity extraction model in PyTorch using <code>bert-base-uncased</code> but when I try to run the model I get this error:</p>
  <h2>Error:</h2>
  <pre><code>Some weights of the model checkpoint at D:\Transformers\bert-entity-extraction\input\bert-base-uncased_L-12_H-768_A-12 were not used when initializing BertModel:    
  ['cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight',   'cls.predictions.transform.LayerNorm.bias', 'cls.seq_relationship.bias',  
   'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.weight',  
   'cls.predictions.bias']  
      - This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
      - This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
  </code></pre>
  <p>I have downloaded the bert model from <a href="https://github.com/google-research/bert" rel="noreferrer">here</a> and the additional files from <a href="https://huggingface.co/bert-base-uncased/tree/main" rel="noreferrer">here</a></p>
  <h2>Code</h2>
  <p>Following is the code for my model:</p>
  <pre><code>import config
  import torch
  import transformers
  import torch.nn as nn
  
  def loss_fn(output, target, mask, num_labels):
  
      lfn = nn.CrossEntropyLoss()
      active_loss = mask.view(-1) == 1
      active_logits = output.view(-1, num_labels)
      active_labels = torch.where(
          active_loss,
          target.view(-1),
          torch.tensor(lfn.ignore_index).type_as(target)
      )
      loss = lfn(active_logits, active_labels)
      return loss
  
  class EntityModel(nn.Module):
      def __init__(self, num_tag, num_pos):
          super(EntityModel, self).__init__()
  
          self.num_tag = num_tag
          self.num_pos = num_pos
          self.bert = transformers.BertModel.from_pretrained(config.BASE_MODEL_PATH)
          self.bert_drop_1 = nn.Dropout(p = 0.3)
          self.bert_drop_2 = nn.Dropout(p = 0.3)
          self.out_tag = nn.Linear(768, self.num_tag)
          self.out_pos = nn.Linear(768, self.num_pos)
  
      def forward(self, ids, mask, token_type_ids, target_pos, target_tag):
          o1, _ = self.bert(ids, 
                            attention_mask = mask,
                            token_type_ids = token_type_ids)
  
          bo_tag = self.bert_drop_1(o1)
          bo_pos = self.bert_drop_2(o1)
  
          tag = self.out_tag(bo_tag)
          pos = self.out_pos(bo_pos)
  
          loss_tag = loss_fn(tag, target_tag, mask, self.num_tag)
          loss_pos = loss_fn(pos, target_pos, mask, self.num_pos)
  
          loss = (loss_tag + loss_pos) / 2
  
          return tag, pos, loss 
  
  print(&quot;model.py run success!&quot;)
  </code></pre>