supported.
torch.distributed backends
Currently, only the "nccl" and "gloo" backends for torch.distributed
are supported on ROCm.
CUDA API to HIP API mappings in C++
Please refer: https://rocmdocs.amd.com/en/latest/Programming_Guides/H
IP_API_Guide.html
NOTE: The CUDA_VERSION macro, cudaRuntimeGetVersion and
cudaDriverGetVersion APIs do not semantically map to the same values
as HIP_VERSION macro, hipRuntimeGetVersion and hipDriverGetVersion
APIs. Please do not use them interchangeably when doing version
checks.
For example: Instead of using
"#if defined(CUDA_VERSION) && CUDA_VERSION >= 11000" to implicitly
exclude ROCm/HIP,
use the following to not take the code path for ROCm/HIP:
"#if defined(CUDA_VERSION) && CUDA_VERSION >= 11000 &&
!defined(USE_ROCM)"
Alternatively, if it is desired to take the code path for ROCm/HIP:
"#if (defined(CUDA_VERSION) && CUDA_VERSION >= 11000) ||
defined(USE_ROCM)"