architectures, especially more recent ones, allow a few truncations of
the intermediate accumulation results to the reduced precision (e.g.,
half-precision). This change is often benign from the perspective of
model convergence, though it may lead to unexpected results (e.g.,
"inf" values when the final result should be be representable in half-
precision). If reduced-precision reductions are problematic, they can
be turned off with
"torch.backends.cuda.matmul.allow_fp16_reduced_precision_reduction =
False"
A similar flag exists for BF16 GEMM operations and is turned off by
default. If BF16 reduced-precision reductions are problematic, they
can be turned off with
"torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction =
False"
For more information see allow_fp16_reduced_precision_reduction and
allow_bf16_reduced_precision_reduction
Reduced Precision FP16 and BF16 GEMMs and Convolutions on AMD Instinct MI200 devices