<h1 dir="auto"><g-emoji class="g-emoji" alias="bug" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f41b.png">🐛</g-emoji> Bug ImportError: cannot import name 'DataCollatorForLanguageModeling' in run_language_modeling.py</h1>
<h2 dir="auto">Information</h2>
<blockquote>
<p dir="auto">Traceback (most recent call last):<br>
File "transformers/examples/run_language_modeling.py", line 29, in <br>
from transformers import (<br>
ImportError: cannot import name 'DataCollatorForLanguageModeling'</p>
</blockquote>
<p dir="auto">Model I am using (Bert, XLNet ...): roberta, Trying to build the model from scratch using the tutorial.</p>
<p dir="auto">Language I am using the model on (English, Chinese ...): English</p>
<p dir="auto">The problem arises when using:</p>
<ul class="contains-task-list">
<li class="task-list-item">
<p dir="auto"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> the official example scripts: (give details below)<br>
transformers/examples/run_language_modeling.py</p>
</li>
<li class="task-list-item">
<p dir="auto"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> my own modified scripts: (give details below)</p>
</li>
</ul>
<p dir="auto">The tasks I am working on is:</p>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> an official GLUE/SQUaD task: (give the name)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> my own task or dataset: (give details below)<br>
My own Data set</li>
</ul>
<h2 dir="auto">To reproduce</h2>
<p dir="auto">Steps to reproduce the behavior:</p>
<p dir="auto">Follow the tutorial <a href="https://huggingface.co/blog/how-to-train" rel="nofollow">here</a></p>

<h2 dir="auto">Expected behavior</h2>

<h2 dir="auto">Environment info</h2>

<ul dir="auto">
<li><code class="notranslate">transformers</code> version: transformers-2.8.0 sacremoses-0.0.41 sentencepiece-0.1.85 tokenizers-0.5.2</li>
<li>Platform:</li>
<li>Python version: 3.x</li>
<li>PyTorch version (GPU?):1.4.0</li>
<li>Using GPU in script?: No</li>
<li>Using distributed or parallel set-up in script?: None</li>
</ul>
<h1 dir="auto"><g-emoji class="g-emoji" alias="bug" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f41b.png">🐛</g-emoji> Bug</h1>
<h2 dir="auto">Information</h2>
<p dir="auto">Model I am using (ALBERT):</p>
<p dir="auto">Language I am using the model on (Sanskrit, Hindi):</p>
<p dir="auto">The problem arises when using:</p>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> the official example scripts: (give details below)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> my own modified scripts: (give details below)</li>
</ul>
<p dir="auto">The tasks I am working on is:</p>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> an official GLUE/SQUaD task: (give the name)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> my own task or dataset: (give details below)</li>
</ul>
<h2 dir="auto">To reproduce</h2>
<p dir="auto">Steps to reproduce the behavior:</p>
<ol dir="auto">
<li>In Google Colab</li>
<li><code class="notranslate">!python /content/transformers/examples/run_language_modeling.py \ --train_data_file /content/corpus/train/full.txt \ --eval_data_file /content/corpus/valid/full_val.txt \ --model_type albert-base-v2 \</code></li>
<li>This worked yesterdy, bbut the latest added <code class="notranslate">DataCollatorForLanguageModeling</code> can't be imported.</li>
</ol>
<p dir="auto">The error i am getting</p>
<p dir="auto"><code class="notranslate">2020-04-22 05:12:25.640328: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 Traceback (most recent call last): File "/content/transformers/examples/run_language_modeling.py", line 29, in &lt;module&gt; from transformers import ( ImportError: cannot import name 'DataCollatorForLanguageModeling'</code></p>
<p dir="auto">So, I checked if it can be imported directly.</p>
<p dir="auto"><code class="notranslate">from transformers import DataCollatorForLanguageModeling</code><br>
ERROR<br>
<code class="notranslate">from transformers import DataCollatorForLanguageModeling ImportError: cannot import name 'DataCollatorForLanguageModeling' </code></p>
<h2 dir="auto">Expected behavior</h2>

<h2 dir="auto">Environment info</h2>

<ul dir="auto">
<li><code class="notranslate">transformers</code> version: 2.8.0</li>
<li>Platform: Colab</li>
<li>Python version:  3.6.9</li>
<li>PyTorch version (GPU?): 1.4.0</li>
<li>Tensorflow version (GPU?): 2.2.0-rc3</li>
<li>Using GPU in script?: Yes</li>
<li>Using distributed or parallel set-up in script?: No</li>
</ul>