title: Using BERT on cpu: module must have its parameters and buffers on device cuda:0 (device_ids[0]) but found one of them on device: cpu
body: <p>I am trying to use my fine-tuned BERT for visualization on my local machine. The model parameters are saved in a file called <code>trained_model.pt</code>. When I try to load and use it, I get the following error:</p>
  <pre><code>import torch
  from pytorch_pretrained_bert import BertTokenizer, BertModel, BertForMaskedLM
  
  # Load pre-trained model tokenizer (vocabulary)
  tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
  
  # Tokenized input
  text = &quot;[CLS] Who was Jim Henson ? [SEP] Jim Henson was a puppeteer [SEP]&quot;
  tokenized_text = tokenizer.tokenize(text)
  
  # Mask a token that we will try to predict back with `BertForMaskedLM`
  masked_index = 8
  tokenized_text[masked_index] = '[MASK]'
  assert tokenized_text == ['[CLS]', 'who', 'was', 'jim', 'henson', '?', '[SEP]', 'jim', '[MASK]', 'was', 'a', 'puppet', '##eer', '[SEP]']
  
  # Convert token to vocabulary indices
  indexed_tokens = tokenizer.convert_tokens_to_ids(tokenized_text)
  # Define sentence A and B indices associated to 1st and 2nd sentences (see paper)
  segments_ids = [0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1]
  
  # Convert inputs to PyTorch tensors
  tokens_tensor = torch.tensor([indexed_tokens])
  segments_tensors = torch.tensor([segments_ids])
  
  # Load pre-trained model (weights)
  model = torch.load('trained_model.pt', map_location=torch.device('cpu'))
  model.eval()
  
  # Predict hidden states features for each layer
  with torch.no_grad():
      encoded_layers, _ = model(tokens_tensor, segments_tensors, output_all_encoded_layers=False)
  </code></pre>
  <hr />
  <pre><code>RuntimeError                              Traceback (most recent call last)
  &lt;ipython-input-32-65859554cc9e&gt; in &lt;module&gt;
        5 # Predict hidden states features for each layer
        6 with torch.no_grad():
  ----&gt; 7     encoded_layers, _ = model(tokens_tensor, segments_tensors, output_all_encoded_layers=False)
  
  /opt/anaconda3/envs/bert/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
      725             result = self._slow_forward(*input, **kwargs)
      726         else:
  --&gt; 727             result = self.forward(*input, **kwargs)
      728         for hook in itertools.chain(
      729                 _global_forward_hooks.values(),
  
  /opt/anaconda3/envs/bert/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py in forward(self, *inputs, **kwargs)
      151         for t in chain(self.module.parameters(), self.module.buffers()):
      152             if t.device != self.src_device_obj:
  --&gt; 153                 raise RuntimeError(&quot;module must have its parameters and buffers &quot;
      154                                    &quot;on device {} (device_ids[0]) but found one of &quot;
      155                                    &quot;them on device: {}&quot;.format(self.src_device_obj, t.device))
  
  RuntimeError: module must have its parameters and buffers on device cuda:0 (device_ids[0]) but found one of them on device: cpu
  </code></pre>
  <p>Can anyone explain me, how this is resolved? ..by the way I get problems with DataParallel <strong>all the time</strong>. Not sure if the error is caused by that, but this feature seems really buggy to me.</p>