title: ValueError: Unrecognized model in ./MRPC/. Should have a `model_type` key in its config.json, or contain one of the following strings in its name
body: <p>Goal: Amend this <a href="https://github.com/microsoft/onnxruntime-inference-examples/blob/main/quantization/notebooks/bert/Bert-GLUE_OnnxRuntime_quantization.ipynb" rel="nofollow noreferrer">Notebook</a> to work with <strong>Albert</strong> and <strong>Distilbert</strong> models</p>
  <p>Kernel: <code>conda_pytorch_p36</code>. I did Restart &amp; Run All, and refreshed file view in working directory.</p>
  <p>Error occurs in <strong>Section 1.2</strong>, only for these 2 new models.</p>
  <p>For filenames etc., I've created a variable used everywhere:</p>
  <pre class="lang-py prettyprint-override"><code>MODEL_NAME = 'albert-base-v2'  # 'distilbert-base-uncased', 'bert-base-uncased'
  </code></pre>
  <p>I replaced imports with:</p>
  <pre class="lang-py prettyprint-override"><code>from transformers import (AutoConfig, AutoModel, AutoTokenizer)
  #from transformers import (BertConfig, BertForSequenceClassification, BertTokenizer,)
  </code></pre>
  <p>As suggested in <a href="https://huggingface.co/docs/transformers/model_doc/auto" rel="nofollow noreferrer">Transformers Documentation - Auto Classes</a>.</p>
  <blockquote>
  <p>Instantiating one of AutoConfig, AutoModel, and AutoTokenizer will directly create a class of the relevant architecture.</p>
  </blockquote>
  <hr />
  <p><strong>Section 1.2:</strong></p>
  <pre class="lang-py prettyprint-override"><code># load model
  model = AutoModel.from_pretrained(configs.output_dir)  # BertForSequenceClassification
  model.to(configs.device)
  
  
  # quantize model
  quantized_model = torch.quantization.quantize_dynamic(
      model, {torch.nn.Linear}, dtype=torch.qint8
  )
  
  #print(quantized_model)
  
  def print_size_of_model(model):
      torch.save(model.state_dict(), &quot;temp.p&quot;)
      print('Size (MB):', os.path.getsize(&quot;temp.p&quot;)/(1024*1024))
      os.remove('temp.p')
  
  print_size_of_model(model)
  print_size_of_model(quantized_model)
  </code></pre>
  <p>Traceback:</p>
  <pre><code>ValueError: Unrecognized model in ./MRPC/. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: imagegpt, qdqbert, vision-encoder-decoder, trocr, fnet, segformer, vision-text-dual-encoder, perceiver, gptj, layoutlmv2, beit, rembert, visual_bert, canine, roformer, clip, bigbird_pegasus, deit, luke, detr, gpt_neo, big_bird, speech_to_text_2, speech_to_text, vit, wav2vec2, m2m_100, convbert, led, blenderbot-small, retribert, ibert, mt5, t5, mobilebert, distilbert, albert, bert-generation, camembert, xlm-roberta, pegasus, marian, mbart, megatron-bert, mpnet, bart, blenderbot, reformer, longformer, roberta, deberta-v2, deberta, flaubert, fsmt, squeezebert, hubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm-prophetnet, prophetnet, xlm, ctrl, electra, speech-encoder-decoder, encoder-decoder, funnel, lxmert, dpr, layoutlm, rag, tapas, splinter, sew-d, sew, unispeech-sat, unispeech, wavlm
  </code></pre>
  <p>Please let me know if there's anything else I can add to post.</p>