Cuda out of memory in loop on second forward pass?
I have the following problem:
model = MyModel()
model.load_state_dict(checkpoint[weights])
model.train()
data, label = get_data() # just take one trainings example
data.cuda()
for i in range(10): # lets predict data 10 times
    output = model(data)
    print(i)

I can do one forward step (i is printed 0) but then I get a Cuda OOM error at output = model(data).
It works if I use with torch.no_grad() but I'd like to train in this forward loop and so I need the gradients later.
Do I somehow have to clear the earlier output or data so it doesn't take up my Vram?
output = model(data.clone()) # this doesn't fix the problem

for i in range(10):
  model.zero_grad() # doesn't work either
  output = model(data)


Ok you have to manually remove the output tensor.
It is kept although the loop starts over again:
This is fixable by
for i in range(10):
  model.zero_grad()
  output = model(data)
  del output

