Changing config and loading Hugging Face model fine-tuned on a downstream task
I am using HuggingFace models for TokenClassification task. I have the following label2id mapping. I am using version 3.3.0 of the library
label2id = {
    &quot;B-ADD&quot;: 4,
    &quot;B-ARRESTED&quot;: 7,
    &quot;B-CRIME&quot;: 2,
    &quot;B-INCIDENT_DATE&quot;: 3,
    &quot;B-SUSPECT&quot;: 9,
    &quot;B-VICTIMS&quot;: 1,
    &quot;B-WPN&quot;: 5,
    &quot;I-ADD&quot;: 8,
    &quot;I-ARRESTED&quot;: 13,
    &quot;I-CRIME&quot;: 11,
    &quot;I-INCIDENT_DATE&quot;: 10,
    &quot;I-SUSPECT&quot;: 14,
    &quot;I-VICTIMS&quot;: 12,
    &quot;I-WPN&quot;: 6,
    &quot;O&quot;: 0
  }

The following scenario works well and the model gets loaded correctly.
from transformers import AutoModelForTokenClassification, AutoTokenizer, AutoConfig

pretrained_model_name = &quot;bert-base-cased&quot;
config = AutoConfig.from_pretrained(pretrained_model_name)

id2label = {y:x for x,y in label2id.items()}
config.label2id = label2id
config.id2label = id2label
config._num_labels = len(label2id)

model = AutoModelForTokenClassification.from_pretrained(pretrained_model_name, config=config)

model

I get the following output. The last layer have been correctly initialized with 15 neurons (numer of token category to predict).
.....................
      (dropout): Dropout(p=0.1, inplace=False)
      (classifier): Linear(in_features=768, out_features=15, bias=True)
    )

but if I changed the pretrained_model_name to &quot;dbmdz/bert-large-cased-finetuned-conll03-english&quot;, I am getting the following error
loading weights file https://cdn.huggingface.co/dbmdz/bert-large-cased-finetuned-conll03-english/pytorch_model.bin from cache at C:\Users\anu10961/.cache\torch\transformers\4b02c1fe04cf7f7e6972536150e9fb329c7b3d5720b82afdac509bd750c705d2.6dcb154688bb97608a563afbf68ba07ae6f7beafd9bd98b5a043cd269fcc02b4
All model checkpoint weights were used when initializing BertForTokenClassification.

All the weights of BertForTokenClassification were initialized from the model checkpoint at dbmdz/bert-large-cased-finetuned-conll03-english.
If your task is similar to the task the model of the checkpoint was trained on, you can already use BertForTokenClassification for predictions without further training.

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-15-2969a8092bf4&gt; in &lt;module&gt;
----&gt; 1 model = AutoModelForTokenClassification.from_pretrained(pretrained_model_name, config=config)

C:\ProgramData\Anaconda3\envs\arcgis183\lib\site-packages\transformers\modeling_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   1372         if type(config) in MODEL_FOR_TOKEN_CLASSIFICATION_MAPPING.keys():
   1373             return MODEL_FOR_TOKEN_CLASSIFICATION_MAPPING[type(config)].from_pretrained(
-&gt; 1374                 pretrained_model_name_or_path, *model_args, config=config, **kwargs
   1375             )
   1376 

C:\ProgramData\Anaconda3\envs\arcgis183\lib\site-packages\transformers\modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   1047                 raise RuntimeError(
   1048                     &quot;Error(s) in loading state_dict for {}:\n\t{}&quot;.format(
-&gt; 1049                         model.__class__.__name__, &quot;\n\t&quot;.join(error_msgs)
   1050                     )
   1051                 )

RuntimeError: Error(s) in loading state_dict for BertForTokenClassification:
    size mismatch for classifier.weight: copying a param with shape torch.Size([9, 1024]) from checkpoint, the shape in current model is torch.Size([15, 1024]).
    size mismatch for classifier.bias: copying a param with shape torch.Size([9]) from checkpoint, the shape in current model is torch.Size([15]).

The only difference I could see is the model dbmdz/bert-large-cased-finetuned-conll03-english is already finetuned on TokenClassification task and it model config has these label2id mappings
label2id = {
    &quot;B-LOC&quot;: 7,
    &quot;B-MISC&quot;: 1,
    &quot;B-ORG&quot;: 5,
    &quot;B-PER&quot;: 3,
    &quot;I-LOC&quot;: 8,
    &quot;I-MISC&quot;: 2,
    &quot;I-ORG&quot;: 6,
    &quot;I-PER&quot;: 4,
    &quot;O&quot;: 0
  }

But still I feel that we can change the last layer of this model and use it for my specific task (although I need to train the model first before using it for inferencing)

Once a part of the model is in the saved pre-trained model, you cannot change its hyperparameters. By setting the pre-trained model and the config, you are saying that you want a model that classifies into 15 classes and that you want to initialize with a model that uses 9 classes and that does not work.
If I understand correctly you want to initialize the underlying BERT from a different classifier. A workaround that can do it is:

Load only the underlying BERT without the classification layer;
Initialize a classification model from scratch;
Replace the randomly initialized BERT in the new classifier with the pre-trained one.

from Transformers import AutoModel, AutoModelForTokenClassification
bert = AutoModel.from_pretrained('dbmdz/bert-large-cased-finetuned-conll03-english')
classifier = AutoModelForTokenClassification.from_config(config)
classifier.bert = bert

