How to clean garbage from CUDA in Pytorch?
I teached my neural nets and realized that even after torch.cuda.empty_cache() and gc.collect() my cuda-device memory is filled. In Colab Notebooks we can see the current variables in memory, but even I delete every variable and clean the garbage gpu-memory is busy.
I heard it's because python garbage collector can't work on cuda-device.
Please explain me, what should I do?

You can do this:
import gc
import torch
gc.collect()
torch.cuda.empty_cache()

