Is there a way to convert a python dictionary with some values on GPU memory to have everything on main memory?
I have a Deep Learning (Using PyTorch) model whose output is given in dictionary format. The dictionary has multiple arrays as values and all these arrays are on GPU memory (torch.tensors with device = 'cuda'). Is there any way to move every array in the dictionary to main memory in one go?
My current way of going about this is to write a loop and re-write the GPU arrays into variables and use those, but that seems quite ineffecient. My essential goal is to further process the outputs in a multi-threaded manner, but since these arrays are on GPU memory mutliple threads cannot access them at once.
Would appreciate any sort of help or suggestions for this!

You can use a dictionary comprehension on the output dictionary out:
out = {k: v.to(device='cpu', non_blocking=True) for k, v in out.items()}

If out has some elements that are not tensors, you can use:
out = {k: v.to(device='cpu', non_blocking=True) if hasattr(v, 'to') else v for k, v in out.items()}


