Image clustering - allocating memory on GPU
I've written this code for image classification by pretrained googlenet:

gnet = models.googlenet(pretrained=True).cuda()

transform = transforms.Compose([transforms.Resize(256), transforms.CenterCrop(32), transforms.ToTensor()])
images = {}
resultDist = {}
i = 1

for f in glob.iglob("/data/home/student/HW3/trainData/train2014/*"):
    print(i)
    i = i + 1
    image = Image.open(f)
    # transform, create batch and get gnet weights
    img_t = transform(image).cuda()
    batch_t = torch.unsqueeze(img_t, 0).cuda()
    try:
        gnet.eval()
        out = gnet(batch_t)
        resultDist[f[-10:-4]] = out
        del out
    except:
        print(img_t.shape)
    del img_t
    del batch_t
    image.close()
    torch.cuda.empty_cache()
    i = i + 1

torch.save(resultDist, '/data/home/student/HW3/googlenetOutput1.pkl')


I deleted all the possible tensors from the GPU after using them, but after about 8000 images from my dataset the GPU is full. I found the problem to be in:

resultDist[f[-10:-4]] = out


The dictionary taking alot of space and I can't delete it because I want to save my data to pkl file.

Since you're not doing backprop wrap your whole loop with a with torch.no_grad(): statement since otherwise a computation graph is created and intermittent results may be stored on the GPU for later application of backprop. This takes a fair amount of space. Also you probably want to save out.cpu() so your results aren't left on the GPU.

...
with torch.no_grad():
    for f in glob.iglob("/data/home/student/HW3/trainData/train2014/*"):
        ...
            resultDist[f[-10:-4]] = out.cpu()
        ...

torch.save(resultDist, '/data/home/student/HW3/googlenetOutput1.pkl')

