[NVIDIA XLA] Support Conv-Bias-Relu6/LeakyRelu fusion in XLA using cuDNN runtime fusion
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this [failed invocation](https://github.com/tensorflow/tensorflow/pull/59614/checks?check_run_id=11205585012) of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request. @kaixih and @philipphack could you both do a quick review please?
We'll remove the Draft state once you take a look. @philipphack could you take a quick look? @philipphack @kaixih could you please take a look and provide feebacks? Hi @Young768 , can you resolve the conflict first? @cheshire what else I need to do on this PR? I added the fix to [outline_cuda_graphs.mlir](https://github.com/tensorflow/tensorflow/pull/59614/files#diff-45654d97055eed7c7d12154ede62d2019af46e8719ef8c2ff886755ced12c387) to fix the cudagraph test since your last approval. Could you please help trigger the tests as well? @cheshire could you please share the log of that internal test feedback/copybara? So I can know what failed the test. @Young768 A test is failing internally in `xla_aot_compile_gpu_test.cc` on [this line](https://cs.opensource.google/tensorflow/tensorflow/+/master:tensorflow/compiler/xla/service/xla_aot_compile_gpu_test.cc;l=238;drc=c0759632b00da7d3709e6927171342450e6fce63). This test has the [`no_oss` tag](https://cs.opensource.google/tensorflow/tensorflow/+/master:tensorflow/compiler/xla/service/BUILD;l=7081;drc=67380fa97b52c0ab186efdff89c6353194c142f4) which is why the GitHub CI passed. To pass the XLA test, you need to add the leakyrelu_alpha field to this hlo https://cs.opensource.google/tensorflow/tensorflow/+/master:tensorflow/compiler/xla/service/xla_aot_compile_test_autotune_results.prototxt;l=26?q=xla_aot_compile_test_&ss=tensorflow%2Ftensorflow

So that the highlighted line becomes 
`  hlo: "(f32[1,1,2,3]{3,2,1,0}, u8[0]{0}) custom-call(f32[2,1,4,4]{3,2,1,0}, f32[2,1,3,2]{3,2,1,0}), window={size=2x3}, dim_labels=bf01_oi01->bf01, custom_call_target=\"__cudnn$convBackwardFilter\", backend_config=\"{\\\"activation_mode\\\":\\\"0\\\",\\\"conv_result_scale\\\":1,\\\"side_input_scale\\\":0,\\\"leakyrelu_alpha\\\":0}\""` @reedwm could you help trigger the CI tests? This was rolled back, unfortunately.

@jlebar do you know the details of what broke and if it's possible to fix and resubmit this PR in a way that will not cause breakages again? > @jlebar do you know the details of what broke and if it's possible to fix and resubmit this PR in a way that will not cause breakages again?

My understanding is, this PR appears to require cudnn with libnvrtc support.  We do not build with libnvrtc enabled today inside Google, so this will be broken for anyone who has relu6 and runs on Ampere at Google.

Until Google supports libnvrtc, Google will need to disable the generation of convs that use libnvrtc.

I believe @Artem-B is working on enabling libnvrtc at Google because his understanding is it's required for cudnn 8.8. @reedwm @jlebar I think nvrtc is built inside cudnn since cudnn 8? Since another merged [PR](https://github.com/tensorflow/tensorflow/pull/57794) has already used the runtime engine. The change enabling libnvrtc should land next week.

I may be completely off base here as I'm not very familiar with the specifics of interaction between XLA, cuDNN, cudnn_frontend and libnvrtc, but the "appears to require cudnn with libnvrtc support" part strikes me as odd.

Libnvrtc availability appears to be an optional feature. TensorRT and cuDNN have been working fine for us without it, even if they may leave some performance on the table. I would assume that any functionality in XLA also should not have the hard dependency on libnvrtc availability either. If the change does depend on libnvrtc, is it possible to query if the required functionality is available and fall back on something less fancy if it is not? What am I missing?

 >  I think nvrtc is built inside cudnn since cudnn 8?

That's certainly not the case for the static variant of cuDNN library. Versions up to 8.7 `dlopen()` libnvrtc at runtime. I believe v8.8 has started to link with libnvrtc statically which is what got me to finally get it working in our internal build.
 @Young768, libnvrtc is now supported in Google according to @jlebar, so we can take this change now.

Since this PR was already merged (and then rolled back), can you create a new PR with this change? @reedwm I *think* we can take this change, but please double-check that this is correct!  :) Good point, once @Young768 creates the PR, I'll manually import it and run the tests that were previously broken before submitting.

@Young768 please CC me on the new PR :)
This PR enables the Conv-Bias-Relu6/LeakyRelu fusion in XLA, which can take advantage of the cuDNN runtime compiled fusion kernels.

A quick benchmark measured that this can bring 6~7% performance improvement.

@kaixih @pjannaty @nluehr 