GPU shows 0 utilization even when tensors and model are mounted on the gpu?
The problem was due to some modules I was importing without ensuring the tensors are mounted on cuda through that part of the pipeline. This had turned into the bottleneck blocking the GPU utilization.