title: How to clear CUDA memory in PyTorch
body: <p>I am trying to get the output of a neural network which I have already trained. The input is an image of the size 300x300. I am using a batch size of 1, but I still get a <code>CUDA error: out of memory</code> error after I have successfully got the output for 25 images.</p>
  <p>I tried <code>torch.cuda.empty_cache()</code>, but this still doesn't seem to solve the problem. Code:</p>
  <pre><code>device = torch.device(&quot;cuda:0&quot; if torch.cuda.is_available() else &quot;cpu&quot;)
  
  train_x = torch.tensor(train_x, dtype=torch.float32).view(-1, 1, 300, 300)
  train_x = train_x.to(device)
  dataloader = torch.utils.data.DataLoader(train_x, batch_size=1, shuffle=False)
  
  right = []
  for i, left in enumerate(dataloader):
      print(i)
      temp = model(left).view(-1, 1, 300, 300)
      right.append(temp.to('cpu'))
      del temp
      torch.cuda.empty_cache()
  </code></pre>
  <p>This <code>for loop</code> runs for 25 times every time before giving the memory error.</p>
  <p>Every time, I am sending a new image in the network for computation. So, I don't really need to store the previous computation results in the GPU after every iteration in the loop. Is there any way to achieve this?</p>