<ul dir="auto">
<li><code class="notranslate">transformers</code> version: 3.0.2</li>
<li>Platform: Linux</li>
<li>Python version: 3.6</li>
<li>PyTorch version (GPU?): 1.4 (GPU)</li>
<li>Using GPU in script?: Yes</li>
<li>Using distributed or parallel set-up in script?: No</li>
</ul>
<h3 dir="auto">Who can help</h3>
<p dir="auto"><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/sshleifer/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/sshleifer">@sshleifer</a></p>
<h2 dir="auto">Information</h2>
<p dir="auto">Model I am using (Bert, XLNet ...): BART</p>
<p dir="auto">The problem arises when using:</p>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> the official example scripts: (give details below)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> my own modified scripts: (give details below)</li>
</ul>
<p dir="auto">The tasks I am working on is:</p>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> an official GLUE/SQUaD task: (give the name)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> my own task or dataset: (give details below)</li>
</ul>
<h2 dir="auto">To reproduce</h2>
<p dir="auto">Steps to reproduce the behavior:</p>
<ol dir="auto">
<li>Running the example script in <a href="https://github.com/huggingface/transformers/tree/master/examples/seq2seq">https://github.com/huggingface/transformers/tree/master/examples/seq2seq</a> (finetune_bart_tiny.sh), I'm getting this warning in the beginning of training. However, the training process is continuing after that.</li>
</ol>
<p dir="auto">Warning:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="finetune.py:245: UserWarning: All learning rates are 0
  warnings.warn(&quot;All learning rates are 0&quot;)
Epoch 1:   0%|      
/home/sajad/anaconda3/lib/python3.6/site-packages/torch/optim/lr_scheduler.py:224: UserWarning: To get the last learning rate computed by the scheduler, please use `get_last_lr()`.
  warnings.warn(&quot;To get the last learning rate computed by the scheduler, &quot;"><pre class="notranslate"><code class="notranslate">finetune.py:245: UserWarning: All learning rates are 0
  warnings.warn("All learning rates are 0")
Epoch 1:   0%|      
/home/sajad/anaconda3/lib/python3.6/site-packages/torch/optim/lr_scheduler.py:224: UserWarning: To get the last learning rate computed by the scheduler, please use `get_last_lr()`.
  warnings.warn("To get the last learning rate computed by the scheduler, "
</code></pre></div>
<h2 dir="auto">Expected behavior</h2>
<p dir="auto">While the training seemingly goes well, I'm wondering if this warning would cause problems, leading to deteriorate model's final performance? As a add-on, I've also incorporated the gradient checkpointing to some computational blocks of BART (modifying <code class="notranslate">modelling_bart.py</code> script a bit). But even w/o incorporating this module, I'm still getting this warning message? Any thoughts of how to solve it?</p>
<p dir="auto">transformers/examples/seq2seq/finetune.py  219row :<br>
if max(scheduler.get_last_lr()) &gt; 0:<br>
warnings.warn("All learning rates are 0")</p>
<p dir="auto">why, is there something i left?</p>