[NVIDIA XLA] FP8 Matmul BiasAdd fusion for high rank input
Hi @wenscarl Can you please resolve conflicts? Thank you! This PR is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you. This PR is not stale. It requires https://github.com/tensorflow/tensorflow/pull/60409 to be merged first. I'll reopen if automatically closed. This PR is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you. @reedwm The top of master seems to fail those tests in `gemm_rewrite_test.cc` which prevents from rebasing.  Hi @reedwm Can you please review this PR ? Thank you! @wenscarl thanks for addressing all my comments! Can you resolve conflicts? > @wenscarl thanks for addressing all my comments! Can you resolve conflicts?

The gemm_rewrite_test did not pass for the current master. Could you take a look? > The gemm_rewrite_test did not pass for the current master. Could you take a look?

It's passing for me. I tried on https://github.com/openxla/xla/commit/d7b3b230abca3bc7adafeff7344aaafc6e6994dc with `bazel test //xla/service/gpu/tests:gemm_rewrite_test` and it passes. What commit are you synced to and what command are you using to run?

 I also tried on the TF repo at 35af49017480ee112fc8c9850862f1eb61d611ee, since this PR is on the TF repo. It still passes for me. > I also tried on the TF repo at [35af490](https://github.com/tensorflow/tensorflow/commit/35af49017480ee112fc8c9850862f1eb61d611ee), since this PR is on the TF repo. It still passes for me.

I tried the same commit, Used `TF_CUDA_CLANG=0` in .bazelrc. BatchRowTransposeFoldCheck, BatchFromMinorDimTransposeIsNotFolded, BatchedInstrLayoutTransposed, BatchedInstrLayoutBatchNotInMinorDim failed.
 I'm also using `TF_CUDA_CLANG=0`. I'm using CUDA 12.1.1 with cuDNN 8.9.1. What version of CUDA/cuDNN are you using? And what's the error message? > I'm also using `TF_CUDA_CLANG=0`. I'm using CUDA 12.1.1 with cuDNN 8.9.1. What version of CUDA/cuDNN are you using? And what's the error message?
Seeing :
```
Mismatch count 4 (10.0000%) in shape f32[2,5,4] (40 elements), abs bound 1e-05, rel bound 1e-05
Top relative error mismatches:
  actual         0.000914607197, expected         0.000930683687, index {1,2,2}, rel error   0.0173, abs error 1.61e-05
```
while testing BatchedInstrLayoutTransposed on Hopper.
 What version of CUDA and cuDNN are you using? > What version of CUDA and cuDNN are you using?

cuda 12.1 + cudnn 8.9.2
 I think it's fine to just increase the test tolerance, since the error is almost within tolerances already. > I think it's fine to just increase the test tolerance, since the error is almost within tolerances already.

Fixes by #61415  @wenscarl can you sync past #61415?
This PR complete the coverage for FP8 matmul when input is high rank tensor. @philipphack 