<h2 dir="auto">Environment info</h2>

<ul dir="auto">
<li><code class="notranslate">transformers</code> version:'3.1.0'</li>
<li>Platform:</li>
<li>Python version:</li>
<li>PyTorch version (GPU?):</li>
<li>Tensorflow version (GPU?):</li>
<li>Using GPU in script?:</li>
<li>Using distributed or parallel set-up in script?:</li>
</ul>
<h3 dir="auto">Who can help</h3>

<h2 dir="auto">Information</h2>
<p dir="auto">Model I am using (Bert, XLNet ...):</p>
<p dir="auto">The problem arises when using:</p>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> the official example scripts: (give details below)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> my own modified scripts: (give details below)</li>
</ul>
<p dir="auto">The tasks I am working on is:</p>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> an official GLUE/SQUaD task: (give the name)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> my own task or dataset: (give details below)</li>
</ul>
<h2 dir="auto">To reproduce</h2>
<p dir="auto">Steps to reproduce the behavior:</p>
<ol dir="auto">
<li></li>
<li></li>
<li></li>
</ol>

<h2 dir="auto">Expected behavior</h2>
<p dir="auto">Hi,</p>
<p dir="auto">Thanks for the great library. I noticed this line being added (<a href="https://github.com/huggingface/transformers/blob/v3.1.0/src/transformers/modeling_bert.py#L190">https://github.com/huggingface/transformers/blob/v3.1.0/src/transformers/modeling_bert.py#L190</a>) in the latest update.</p>
<p dir="auto">It breaks checkpoints that were saved when this line wasn't there.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="	Missing key(s) in state_dict: &quot;generator_model.electra.embeddings.position_ids&quot;, &quot;discriminator_model.electra.embeddings.position_ids&quot;. "><pre class="notranslate"><code class="notranslate">	Missing key(s) in state_dict: "generator_model.electra.embeddings.position_ids", "discriminator_model.electra.embeddings.position_ids". 
</code></pre></div>