f = torch.randn(2).cuda(cuda2)
       # d.device, e.device, and f.device are all device(type='cuda', index=2)
Checking for HIP
Whether you are using PyTorch for CUDA or HIP, the result of calling
"is_available()" will be the same. If you are using a PyTorch that has
been built with GPU support, it will return True. If you must check
which version of PyTorch you are using, refer to this example below:
if torch.cuda.is_available() and torch.version.hip:
       # do something specific for HIP
   elif torch.cuda.is_available() and torch.version.cuda:
       # do something specific for CUDA
TensorFloat-32(TF32) on ROCm
TF32 is not supported on ROCm.
Memory management
PyTorch uses a caching memory allocator to speed up memory
allocations. This allows fast memory deallocation without device
synchronizations. However, the unused memory managed by the allocator
will still show as if used in "rocm-smi". You can use