CUDA out of memory runtime error, anyway to delete pytorch "reserved memory"
Like many othersm I'm getting a Runtime error of Cuda out of memory, but for some reason pytorch has reserved a large amount of it.
RuntimeError: CUDA out of memory. Tried to allocate 2.00 MiB (GPU 0; 6.00 GiB total capacity; 4.31 GiB already allocated; 844.80 KiB free; 4.71 GiB reserved in total by PyTorch)
I've tried the torch.cuda.empy_cache(), but this isn't working either and none of the other CUDA out of memory posts have helped me either.
When I've checked my gpu usage(nvidia-smi) before running my python program, it is plenty free.

From the given description it seems that the problem is not allocated memory by Pytorch so far before the execution but cuda ran out of memory while allocating the data that means the 4.31GB got already allocated (not cached) but failed to allocate the 2MB last block.
Possible solution already worked for me, is to decrease the batch size, hope that helps!
