[oneDNN] Make oneDNN default on Windows
> Thank you very much! :)

Thank you very much for your guidance and approving the PR.  2 Windows tests failed internally. Could you please help take a look? This probably happened because we ran on older CPUs that don't have AVX512_BF16.

`//tensorflow/core/grappler/optimizers:auto_mixed_precision_test_cpu`
```
==================== Test output for //tensorflow/core/grappler/optimizers:auto_mixed_precision_test_cpu:
2023-09-18 19:41:58.740895: I [tensorflow/core/util/port.cc:113](https://cs.corp.google.com/piper///depot/google3/tensorflow/core/util/port.cc?l=113&cl=566379082)] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
[==========] Running 5 tests from 1 test suite.
...
[ RUN      ] AutoMixedPrecisionMklTest.InferFollowUpStreamAllow
2023-09-18 19:41:58.891920: I tensorflow/core/grappler/clusters/single_machine.cc:361] Starting new session
2023-09-18 19:41:58.938031: I tensorflow/core/grappler/optimizers/auto_mixed_precision.cc:2254] Converted 3/7 nodes to bfloat16 precision using 0 cast(s) to bfloat16 (excluding Const and Variable casts)
2023-09-18 19:41:58.957392: W tensorflow/core/framework/op_kernel.cc:1839] OP_REQUIRES failed at mkl_conv_ops.cc:1096 : ABORTED: Operation received an exception:Status: 3, message: could not create a primitive descriptor for a convolution forward propagation primitive, in file tensorflow/core/kernels/mkl/mkl_conv_ops.cc:1093
2023-09-18 19:41:58.960676: W tensorflow/core/framework/op_kernel.cc:1839] OP_REQUIRES failed at mkl_conv_ops.cc:1096 : ABORTED: Operation received an exception:Status: 3, message: could not create a primitive descriptor for a convolution forward propagation primitive, in file tensorflow/core/kernels/mkl/mkl_conv_ops.cc:1093
2023-09-18 19:41:58.961610: F tensorflow/core/grappler/utils/grappler_test.cc:124] Non-OK-status: session->Run(run_options, inputs, node_names, node_names, &output_tensors, nullptr) status: ABORTED: Operation received an exception:Status: 3, message: could not create a primitive descriptor for a convolution forward propagation primitive, in file tensorflow/core/kernels/mkl/mkl_conv_ops.cc:1093
	 [[{{node infer}}]]
*** Received signal 22 ***
*** BEGIN STACK TRACE POINTERS ***
```

`//tensorflow/core/grappler/optimizers:remapper_test_cpu`
```
==================== Test output for //tensorflow/core/grappler/optimizers:remapper_test_cpu:
2023-09-18 19:41:52.977040: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
[==========] Running 51 tests from 13 test suites.
...
[ RUN      ] RemapperFuseConvWithBias.Conv2D_BF16
2023-09-18 19:41:55.405579: W tensorflow/core/framework/op_kernel.cc:1839] OP_REQUIRES failed at mkl_conv_ops.cc:1096 : ABORTED: Operation received an exception:Status: 3, message: could not create a primitive descriptor for a convolution forward propagation primitive, in file tensorflow/core/kernels/mkl/mkl_conv_ops.cc:1093
2023-09-18 19:41:55.406251: F tensorflow/core/grappler/utils/grappler_test.cc:124] Non-OK-status: session->Run(run_options, inputs, node_names, node_names, &output_tensors, nullptr) status: ABORTED: Operation received an exception:Status: 3, message: could not create a primitive descriptor for a convolution forward propagation primitive, in file tensorflow/core/kernels/mkl/mkl_conv_ops.cc:1093
	 [[{{node bias_add}}]]
*** Received signal 22 ***
*** BEGIN STACK TRACE POINTERS ***
...
```

 If you don't have older CPUs, I believe you can run with the env var `ONEDNN_MAX_CPU_ISA=AVX2`. (If setting in the shell doesn't work, try passing `--test_env=ONEDNN_MAX_CPU_ISA=AVX2` to bazel) > If you don't have older CPUs, I believe you can run with the env var `ONEDNN_MAX_CPU_ISA=AVX2`. (If setting in the shell doesn't work, try passing `--test_env=ONEDNN_MAX_CPU_ISA=AVX2` to bazel)

The above environment setup worked and we were able to replicate the issue, looking into it @mraunak, oneDNN [requires Intel AVX-512 instruction set](https://oneapi-src.github.io/oneDNN/dev_guide_data_types.html#intel-r-architecture-processors) for bfloat16 functionality. You'll need to add guards for older instruction sets, like Intel AVX2.
 > @mraunak, oneDNN [requires Intel AVX-512 instruction set](https://oneapi-src.github.io/oneDNN/dev_guide_data_types.html#intel-r-architecture-processors) for bfloat16 functionality. You'll need to add guards for older instruction sets, like Intel AVX2.

Hi @penpornk @vpirogov, the test cases have been resolved. Please review and let me know if any further change is needed. Thank you very much. Thank you for approving the changes. sorry, I again posted a commit to ensure no failures of models during the runtime. 2 tests still failed in [Ubuntu CPU](https://source.cloud.google.com/results/invocations/a79e4ba7-e264-495a-9b8b-fe6147474e2c/log)
1. `//tensorflow/python/grappler:auto_mixed_precision_test_cpu`
```
[ RUN      ] AutoMixedPrecisionTest.test_conv3d1 ('mkl')
2023-09-21 22:46:21.978741: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2023-09-21 22:46:22.033759: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:386] MLIR V1 optimization pass is not enabled
2023-09-21 22:46:22.061896: I tensorflow/core/grappler/optimizers/auto_mixed_precision.cc:2330] Running auto_mixed_precision_onednn_bfloat16 graph optimizer
2023-09-21 22:46:22.062308: I tensorflow/core/grappler/optimizers/auto_mixed_precision.cc:1506] No allowlist ops found, nothing to do
2023-09-21 22:46:22.068265: I tensorflow/core/grappler/optimizers/auto_mixed_precision.cc:2330] Running auto_mixed_precision_onednn_bfloat16 graph optimizer
2023-09-21 22:46:22.068735: I tensorflow/core/grappler/optimizers/auto_mixed_precision.cc:2258] Converted 3/19 nodes to bfloat16 precision using 0 cast(s) to bfloat16 (excluding Const and Variable casts)
2023-09-21 22:46:22.074911: E ./tensorflow/core/graph/mkl_graph_util.h:184] oneDNN BFloat16 support are only on platforms with AVX512. Falling back to default implementation if present.
2023-09-21 22:46:22.100372: I tensorflow/core/grappler/optimizers/auto_mixed_precision.cc:2330] Running auto_mixed_precision_onednn_bfloat16 graph optimizer
2023-09-21 22:46:22.100651: I tensorflow/core/grappler/optimizers/auto_mixed_precision.cc:1506] No allowlist ops found, nothing to do
2023-09-21 22:46:22.106529: I tensorflow/core/grappler/optimizers/auto_mixed_precision.cc:2330] Running auto_mixed_precision_onednn_bfloat16 graph optimizer
2023-09-21 22:46:22.106998: I tensorflow/core/grappler/optimizers/auto_mixed_precision.cc:2258] Converted 3/19 nodes to bfloat16 precision using 0 cast(s) to bfloat16 (excluding Const and Variable casts)
/b/f/w/bazel-out/k8-opt/bin/tensorflow/python/grappler/auto_mixed_precision_test_cpu.runfiles/org_tensorflow/tensorflow/python/framework/test_util.py:3109: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
  return np.array(a)
INFO:tensorflow:time(__main__.AutoMixedPrecisionTest.test_conv3d1 ('mkl')): 0.17s
I0921 22:46:22.148852 140329108536320 test_util.py:2574] time(__main__.AutoMixedPrecisionTest.test_conv3d1 ('mkl')): 0.17s
[  FAILED  ] AutoMixedPrecisionTest.test_conv3d1 ('mkl')
...
```

2. `//tensorflow/python/grappler:remapper_test_cpu`
```
======================================================================
FAIL: test_matmul_biasadd_activation_fusion1 ('mkl') (__main__.RemapperTest)
RemapperTest.test_matmul_biasadd_activation_fusion1 ('mkl')
decorated('mkl')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/b/f/w/bazel-out/k8-opt/bin/tensorflow/python/grappler/remapper_test_cpu.runfiles/absl_py/absl/testing/parameterized.py", line 318, in bound_param_test
    return test_method(self, testcase_params)
  File "/b/f/w/bazel-out/k8-opt/bin/tensorflow/python/grappler/remapper_test_cpu.runfiles/org_tensorflow/tensorflow/python/framework/test_util.py", line 1749, in decorated
    return f(self, *args, **kwargs)
  File "/b/f/w/bazel-out/k8-opt/bin/tensorflow/python/grappler/remapper_test_cpu.runfiles/org_tensorflow/tensorflow/python/framework/test_util.py", line 2264, in decorated
    return func(self, *args, **kwargs)
  File "/b/f/w/bazel-out/k8-opt/bin/tensorflow/python/grappler/remapper_test_cpu.runfiles/org_tensorflow/tensorflow/python/framework/test_util.py", line 2436, in decorated
    f(self, *args, **kwargs)
  File "/b/f/w/bazel-out/k8-opt/bin/tensorflow/python/grappler/remapper_test_cpu.runfiles/org_tensorflow/tensorflow/python/grappler/remapper_test.py", line 220, in test_matmul_biasadd_activation_fusion
    graph = self._VerifyValues(out, precision != dtypes.float32, fused_op,
  File "/b/f/w/bazel-out/k8-opt/bin/tensorflow/python/grappler/remapper_test_cpu.runfiles/org_tensorflow/tensorflow/python/grappler/remapper_test.py", line 149, in _VerifyValues
    self.assertTrue(found_fused_op)
AssertionError: False is not true

----------------------------------------------------------------------
``` @penpornk - These tests were failing earlier on your internal Windows running on Broadwell as oneDNN is becoming enabled by default for it. 
With this change, we are disabling BF16 tests on CPUs with only AVX2 or earlier ISAs for which oneDNN doesn't have BF16 support.  OneDNN has emulation support for AVX512 systems without BF16. 
Tests are now disabled on both Linux and Windows. This should not affect Linux systems as oneDNN is not enabled by default and the tests will get bypassed Hi @penpornk, we are working on it and will have an update ASAP. To clean up the PR, we will split the changes into two PRs. This PR #61878 will have changes only to make oneDNN default on Windows. A separate PR which is in progress will be submitted to address all the test case @mraunak Sounds good. Thank you very much for the update! :) Hi @penpornk, this PR https://github.com/tensorflow/tensorflow/pull/61878  have following changes
1.  Make oneDNN default on Windows
2.  Update the 'Major Feature and Improvements' section of the release notes for TensorFlow 2.15.0 with the addition of a new feature of making oneDNN optimization default on the Windows Platform.

Another PR https://github.com/tensorflow/tensorflow/pull/62011 handles the test case failures on older CPU machines. 

Please review and let us know if any further change is needed. Hi @cantonios the above feedback/copybara check failure has been addressed in the PR https://github.com/tensorflow/tensorflow/pull/62011. Request you to re-run the check and merge if the check passes
This PR will make oneDNN optimization default on the Windows platform.