title: Why "CUDA out of memory" if I am updating the same tensors repeatedly?
body: <p>I am using some fixed-size tensors in my code. You can check the code excerpt below</p>
  <pre><code>x = torch.randn(100, 90, 90)
  w = torch.randn(768, 100)  
  
  size = x.size()
  
  for row_index in range(len(w)):
      wr = w[row_index] 
  
      wr = wr.unsqueeze(-1).unsqueeze(-1).expand(size)
      
      x = wr * x
  </code></pre>
  <p>The loop executes for some iterations (approximately 100) and posts the error of &quot;RuntimeError: CUDA out of memory.&quot;</p>
  <p>I am updating them in a loop. I am not using any new tensors; I am updating the same tensors. So, the space should not increase, right? Then why is the error occurring?</p>