title: How to free gpu memory by deleting tensors?
body: <p>Suppose I create a tensor and put it on the GPU and don't need it later and want to free the GPU memory allocated to it; How do I do it?</p>
  
  <pre class="lang-py prettyprint-override"><code>import torch
  a=torch.randn(3,4).cuda() # nvidia-smi shows that some mem has been allocated.
  # do something
  # a does not exist and nvidia-smi shows that mem has been freed.
  </code></pre>
  
  <p>I have tried:</p>
  
  <ol>
  <li><code>del a</code> </li>
  <li><code>del a; torch.cuda.empty_cache()</code></li>
  </ol>
  
  <p>But none of them work.</p>