My model memory size keeps decreasing and isn't clearing. And as a result, I get a CUDA memory issue
I'm trying to run a deep network in CUDA/Pytorch. But I keep getting a GPU issue that tells me I'm out of memory in in my GPU as follows:

CUDA out of memory. Tried to allocate 16.00 MiB (GPU 0; 11.17 GiB total capacity; 10.62 GiB already allocated; 14.81 MiB free; 10.63 GiB reserved in total by PyTorch) If reserved memory is &gt;&gt; allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

And I get the same sort of error even with an HPC, just that the numbers are bigger.
As far as I can tell, my model is fine and as expected. (My professors told us we can run CIFAR with &lt; 5 M parameters and my model has 4.7 M parameters).
I create data loaders as follows:
transform=transforms.Compose([transforms.ToTensor(), 
                             transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),
                             transforms.CenterCrop(32),
                             transforms.RandomRotation(15)
                             ])
#Augmentation helps prevent overfitting. We use Bilinear and not the default (nearest) since Bilinear is a more efficient technique


batch_size=128

train=datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)
test=datasets.CIFAR10(root='./data',train=False,download=True,transform=transform)

#train=train.type(torch.IntTensor)
#train=train.to(uint8)


train_size=int(len(train))
test_size=int(len(test))

train_idx=list(range(train_size))
test_idx=list(range(test_size))

split=int(np.floor(0.77*train_size))

train_idx, val_idx=train_idx[:split] , train_idx[split:]

train_sampler = SubsetRandomSampler(train_idx)
val_sampler = SubsetRandomSampler(val_idx)


train_loader=DataLoader(train,batch_size=batch_size,num_workers=1,sampler=train_sampler)
valid_loader=DataLoader(train,batch_size=batch_size,num_workers=1,sampler=val_sampler)

test_loader=DataLoader(test,batch_size=batch_size,shuffle=False, num_workers=2)


So I split it into a train and test set that are split in a 70/30 fashion.
To run my epochs:
for e in range(epochs):
    running_loss=0.0
    v_loss=0.0
    correct=0
    v_correct=0
    for i,data in enumerate(train_loader):
      print(&quot;Ran a batch&quot;)
      x,y=data
      x=x.to(device)
      y=y.to(device)
      optimizer.zero_grad()
      y_hat=model(x)
      loss=criterion(y_hat,y) #Comparing prediction to actual output
      loss.backward
      optimizer.step()
      running_loss+=loss
      pred=y_hat.argmax(axis=0) #The index associated with correct prediction
      correct+=y[pred] #Either they match and correct. Or different and wrong
      GPUtil.showUtilization()
      del x
      del y
      GPUtil.showUtilization()
      gc.collect() 
      torch.cuda.empty_cache()
      GPUtil.showUtilization()
    print(&quot;end of epoch&quot;)    

In each epoch, I load a batch that I feed to my model. Unfortunatley, it seems like my model can never even train a full batch, that is despite constantly deleting my cache and tensors.
When I look at memory, while GPU utilisation stays balanced and doesn't shoot up, it seems like the memory does and I don't know why that's the case/
From this error, I can see that the memory utilized never clears and it only keeps increasing.

| ID | GPU | MEM |
|  0 | 57% | 76% |

....


|  0 | 68% | 100% |
| ID | GPU | MEM  |

When I look at this problem online, I'm told that decreasing my batch size will help with CUDA running out of memory.
but even when my batch size is 1, the memory keeps growing and causing issues.
As a result, I'm not sure how to solve this problem. I can't tell what's going wrong here.

Maybe typo, but there should be loss.backward() like method calling and loss.backward by itself makes nothing. Also when computing running_loss call .item() method on loss, so full line looks like this running_loss += loss.item(). In discussion there is an answer on why it should reduce memory usage (...cumulative loss will hold reference to each individual loss, which in turn holds references to each of the computation nodes that computed it...).
