title: Pytorch: RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one
body: <p>I am training a multilingual bert model for a sentiment classification task. I have 2 GPUs on 1 Machine so I am using Huggingface <code>Accelerator</code> for distributed training. But when I run the code it throws a Runtime Error.</p>
  <h2>Model</h2>
  <pre><code>class BERTModel(nn.Module):
      def __init__(self):
          super(BERTModel, self).__init__()
          self.bert = transformers.BertModel.from_pretrained(&quot;bert-base-multilingual-uncased&quot;)
          self.bert_drop = nn.Dropout(0.3)
          self.out = nn.Linear(768 * 2, 1) # *2 since we have 2 pooling layers
  
      def forward(self, ids, mask, token_type_ids):
          o1, _ = self.bert(
              ids, 
              attention_mask=mask, 
              token_type_ids=token_type_ids
          )
          
          mean_pooling = torch.mean(o1, 1)
          max_pooling, _ = torch.max(o1, 1)
          cat = torch.cat((mean_pooling, max_pooling), 1)
          
          bo = self.bert_drop(cat)
          output = self.out(bo)
          return output
  </code></pre>
  <h2>Train Function</h2>
  <pre><code>def train_fn(data_loader, model, optimizer, scheduler):
      
          
          &quot;&quot;&quot;
          Training Function for the Model
          
          parameters: data_loader - PyTorch DataLoader
                      model - The Model to be used for training
                      optimizer - The Optimizer to be used for training
                      scheduler - The Learning Rate Scheduler 
                      
          returns: None
          &quot;&quot;&quot;
          accelerator = Accelerator()
  
          model, optimizer, data_loader = accelerator.prepare(model, optimizer, data_loader)
          model.train()
  
          for bi, d in tqdm(enumerate(data_loader), total=len(data_loader)):
              ids = d[&quot;ids&quot;]
              token_type_ids = d[&quot;token_type_ids&quot;]
              mask = d[&quot;mask&quot;]
              targets = d[&quot;targets&quot;]
  
              
              ids = ids.to(torch.long)
              token_type_ids = token_type_ids.to(torch.long)
              mask = mask.to(torch.long)
              targets = targets.to(torch.float)
  
              optimizer.zero_grad()
              outputs = model(ids=ids, mask=mask, token_type_ids=token_type_ids)
  
              loss = loss_fn(outputs, targets)
              
              if bi % 1000 == 0:
                  print(f&quot;bi={bi}, loss={loss}&quot;)
  
              accelerator.backward(loss)
              optimizer.step()
              scheduler.step()
  </code></pre>
  <h2>Error</h2>
  <pre><code>---------------------------------------------------------------------------
  Exception                                 Traceback (most recent call last)
  &lt;timed exec&gt; in &lt;module&gt;
  
  /opt/conda/lib/python3.6/site-packages/accelerate/notebook_launcher.py in notebook_launcher(function, args, num_processes, use_fp16, use_port)
      107             try:
      108                 print(f&quot;Launching a training on {num_processes} GPUs.&quot;)
  --&gt; 109                 start_processes(launcher, nprocs=num_processes, start_method=&quot;fork&quot;)
      110             finally:
      111                 # Clean up the environment variables set.
  
  /opt/conda/lib/python3.6/site-packages/torch/multiprocessing/spawn.py in start_processes(fn, args, nprocs, join, daemon, start_method)
      156 
      157     # Loop on join until it returns True or raises an exception.
  --&gt; 158     while not context.join():
      159         pass
      160 
  
  /opt/conda/lib/python3.6/site-packages/torch/multiprocessing/spawn.py in join(self, timeout)
      117         msg = &quot;\n\n-- Process %d terminated with the following error:\n&quot; % error_index
      118         msg += original_trace
  --&gt; 119         raise Exception(msg)
      120 
      121 
  
  Exception: 
  
  -- Process 1 terminated with the following error:
  Traceback (most recent call last):
    File &quot;/opt/conda/lib/python3.6/site-packages/torch/multiprocessing/spawn.py&quot;, line 20, in _wrap
      fn(i, *args)
    File &quot;/opt/conda/lib/python3.6/site-packages/accelerate/utils.py&quot;, line 274, in __call__
      self.launcher(*args)
    File &quot;&lt;timed exec&gt;&quot;, line 276, in run
    File &quot;&lt;timed exec&gt;&quot;, line 74, in train_fn
    File &quot;/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py&quot;, line 726, in _call_impl
      result = self.forward(*input, **kwargs)
    File &quot;/opt/conda/lib/python3.6/site-packages/torch/cuda/amp/autocast_mode.py&quot;, line 135, in decorate_autocast
      return func(*args, **kwargs)
    File &quot;/opt/conda/lib/python3.6/site-packages/torch/nn/parallel/distributed.py&quot;, line 585, in forward
      self.reducer.prepare_for_backward([])
  
  RuntimeError: Expected to have finished reduction in the prior iteration before 
  starting a new one. This error indicates that your module has parameters that 
  were not used in producing loss. You can enable unused parameter detection by (1) 
  passing the keyword argument `find_unused_parameters=True` to 
  `torch.nn.parallel.DistributedDataParallel`; (2) making sure all `forward`
   function outputs participate in calculating loss. If you already have done the 
  above two steps, then the distributed data parallel module wasn't able to locate
   the output tensors in the return value of your module's `forward` function. 
  Please include the loss function and the structure of the return value of 
  `forward` of your module when reporting this issue (e.g. list, dict, iterable).
  </code></pre>