Activation function of a Dense hidden layer not getting invoked.
NOTE: 
When I downgraded tf from 2.13.0 to 2.12.0, the code worked as expected.

The output with tf 2.12.0. The final decoder layer is outputting values [0.5, 0.5, 0.5] as expected.

```
Python 3.8.17 (default, Jul  5 2023, 15:35:58) 
[Clang 14.0.6 ]
Python Platform: macOS-13.4-arm64-arm-64bit
Tensor Flow Version: 2.12.0
GPU is NOT AVAILABLE
2023-08-21 00:18:53.489119: W tensorflow/tsl/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
1/1 [==============================] - 0s 25ms/step
[[0.5 0.5 0.5]]
``` Hi @therealsachin ,

I have tested the provided code with Tf2.13v on both Linux and Macos and both are generating same output for me.

Please refer the result on colab [gist](https://colab.sandbox.google.com/gist/SuryanarayanaY/fea0b501ffcac2bc214e59a405cfbd48/61650_linux-tf2-13v.ipynb#scrollTo=NEyqZ0JD0pxk).

Please refer the attached logs on Macos.

```
(tf-metal) suryanarayanay-macbookpro:Downloads suryanarayanay$ python 61650_macos.py
Python 3.9.16 (main, Mar  8 2023, 04:29:24) 
[Clang 14.0.6 ]
Python Platform: macOS-13.5-arm64-arm-64bit
Tensor Flow Version: 2.13.0
GPU is available
Metal device set to: Apple M1 Pro

systemMemory: 16.00 GB
maxCacheSize: 5.33 GB

1/1 [==============================] - 0s 320ms/step
[[0.5 0.5 0.5]]
(tf-metal) suryanarayanay-macbookpro:Downloads suryanarayanay$ python 61650_macos.py
Python 3.9.16 (main, Mar  8 2023, 04:29:24) 
[Clang 14.0.6 ]
Python Platform: macOS-13.5-arm64-arm-64bit
Tensor Flow Version: 2.13.0
GPU is available
Metal device set to: Apple M1 Pro

systemMemory: 16.00 GB
maxCacheSize: 5.33 GB

1/1 [==============================] - 0s 50ms/step
[[0.5 0.5 0.5]]
(tf-metal) suryanarayanay-macbookpro:Downloads suryanarayanay$ cat 61650_macos.py 
# -*- coding: utf-8 -*-
"""61650_macos(TF2.13v).ipynb

Automatically generated by Colaboratory.

Original file is located at
    https://colab.research.google.com/drive/1pHTvZjMFeqpRTuX_ShlFwGfDOUQl3AId?resourcekey=0-0g-rwvb7Fa9-4Kuw0eIlRA
"""

# !pip install tensorflow==2.13.*

import tensorflow as tf
import tensorflow.keras
import tensorflow as tf
import platform
import sys
from tensorflow.keras.layers import Input, Dense, Layer
from tensorflow.keras.models import Model

# Print versions:
print(f"Python {sys.version}")
print(f"Python Platform: {platform.platform()}")
print(f"Tensor Flow Version: {tf.__version__}")
gpu = len(tf.config.list_physical_devices('GPU'))>0
print("GPU is", "available" if gpu else "NOT AVAILABLE")

# Setup input
import numpy as np
X_check = np.array([[1, 0, 0]])

# Setup autoencoder model
input_layer = Input(shape=(X_check.shape[1]))
bottleneck = Dense(2, activation='relu', name='bottleneck')(input_layer)
output = Dense(X_check.shape[1], activation='sigmoid', name='output')(bottleneck)
autoencoder = Model(input_layer, output)

# Set encoder layer weights to all negative.
layer = autoencoder.layers[1]
weights = np.array([[-1, -1],[-1, -1], [-1, -1]])
biases = np.array([0, 0])
layer.set_weights([weights, biases])

# create encoder model.
encoder = Model(input_layer, bottleneck)

# create decoder model.
decoder_input = Input(shape=(2,), name='decoder_input')
decoder_layer = autoencoder.layers[-1]
decoder = Model(decoder_input, decoder_layer(decoder_input))

# Run auto-encoder, with [1, 0, 0], since encoder has all negative weights,
# and has 'relu' activation o/p of enocder should all be zeros. And that being
# the input of next sigmod we should get output [0.5, 0.5, 0.5]
output_data = autoencoder.predict(X_check)
print(output_data)(tf-metal) suryanarayanay-macbookpro:Downloads suryanarayanay$ 
``` Hi @SuryanarayanaY ,

FYI. I am running on Apple M2 Max, 64 GB, macOS Ventura Version 13.4 (22F66).

I can replicate the issue on my machine. Is there anything that I can do to debug this more? Like, run all tests locally and see if any of the tests fails? Or do something else.

When I set to eager execution to true the bug does not manifest. So I am guessing this has something to do with the execution graph optimization.  Hi @therealsachin ,

I have disable V2 behaviour using `tf.compat.v1.disable_v2_behavior()`to disable eager execution and added `print(tf.executing_eagerly())` at last. Then executed the code again it's not replicating reported behaviour.Please refer attached logs.

```
(base) suryanarayanay-macbookpro:Downloads suryanarayanay$ python 61650_graph_mode_macos_tf2_13v.py
WARNING:tensorflow:From /Users/suryanarayanay/miniconda/lib/python3.10/site-packages/tensorflow/python/compat/v2_compat.py:107: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
Python 3.10.9 (main, Jan 11 2023, 09:18:18) [Clang 14.0.6 ]
Python Platform: macOS-13.5.1-arm64-arm-64bit
Tensor Flow Version: 2.13.0
GPU is available
Metal device set to: Apple M1 Pro

systemMemory: 16.00 GB
maxCacheSize: 5.33 GB

2023-09-06 14:54:16.368497: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:303] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2023-09-06 14:54:16.368700: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:269] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)
2023-09-06 14:54:16.371797: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:375] MLIR V1 optimization pass is not enabled
2023-09-06 14:54:16.379867: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.
2023-09-06 14:54:16.397685: W tensorflow/c/c_api.cc:304] Operation '{name:'output/kernel/Assign' id:33 op device:{requested: '', assigned: ''} def:{{{node output/kernel/Assign}} = AssignVariableOp[_has_manual_control_dependencies=true, dtype=DT_FLOAT, validate_shape=false](output/kernel, output/kernel/Initializer/random_uniform)}}' was changed by setting attribute after it was run by a session. This mutation will have no effect, and will trigger an error in the future. Either don't modify nodes after running them or create a new session.
2023-09-06 14:54:16.398514: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.
2023-09-06 14:54:16.469451: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.
/Users/suryanarayanay/miniconda/lib/python3.10/site-packages/keras/src/engine/training_v1.py:2359: UserWarning: `Model.state_updates` will be removed in a future version. This property should not be used in TensorFlow 2.0, as `updates` are applied automatically.
  updates=self.state_updates,
2023-09-06 14:54:16.478393: W tensorflow/c/c_api.cc:304] Operation '{name:'output/Sigmoid' id:44 op device:{requested: '', assigned: ''} def:{{{node output/Sigmoid}} = Sigmoid[T=DT_FLOAT, _has_manual_control_dependencies=true](output/BiasAdd)}}' was changed by setting attribute after it was run by a session. This mutation will have no effect, and will trigger an error in the future. Either don't modify nodes after running them or create a new session.
2023-09-06 14:54:16.479109: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.
[[0.5 0.5 0.5]]
False
(base) suryanarayanay-macbookpro:Downloads suryanarayanay$ 
```
The bug doesn't manifest without Eager execution also. Did you tried with any alternate method? 

Thanks! This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61650">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61650">No</a>

### Issue type

Bug

### Have you reproduced the bug with TensorFlow Nightly?

No

### Source

binary

### TensorFlow version

v2.13.0-rc2-7-g1cb1a030a62 2.13.0

### Custom code

Yes

### OS platform and distribution

MacOS 13.4, MacBook Pro M2 Max

### Mobile device

_No response_

### Python version

3.8.17

### Bazel version

_No response_

### GCC/compiler version

_No response_

### CUDA/cuDNN version

_No response_

### GPU model and memory

_No response_

### Current behavior?

Issue:
In the given auto-encoder setup, the encoder layers activation function (relu) is not getting invoked.

1. We create a simple auto-encoder, with Input size 3, hidden size 2, and output back to 3.
2. The activation function of the encoder layer is set a relu.
3. The weights of the encoder layers are all made negative. Idea is, if input is +ve, all the neutrons will have negative value and relu will o/p zero.
4. Give input as [1, 0, 0].
5. We expect the final decoder o/p layer, which has sigmoid activation, to o/p all [0.5, 0.5, 0.5] as the input to this layer from the encoder should have been [0, 0, 0]. 
6. But we find that is not the case, which clearly shows that 'relu' activation of the hidden layer is not getting invoked.

Installation:
pip install tensorflow-macos
pip install tensorflow-metal

### Standalone code to reproduce the issue

```shell
# https://colab.research.google.com/drive/14KKrdiBg8FT2cdUC5pjqJHi5S3BkTOHi?usp=sharing
# The above colab will run fine, but the same code on Mac with the said config has issue.
# Copying the code here for quick reference.

import tensorflow as tf    
import tensorflow.keras
import tensorflow as tf
import platform
import sys
from tensorflow.keras.layers import Input, Dense, Layer
from tensorflow.keras.models import Model

# Print versions:
print(f"Python {sys.version}")
print(f"Python Platform: {platform.platform()}")
print(f"Tensor Flow Version: {tf.__version__}")
gpu = len(tf.config.list_physical_devices('GPU'))>0
print("GPU is", "available" if gpu else "NOT AVAILABLE")

# Setup input
import numpy as np
X_check = np.array([[1, 0, 0]])

# Setup autoencoder model
input_layer = Input(shape=(X_check.shape[1]))
bottleneck = Dense(2, activation='relu', name='bottleneck')(input_layer)
output = Dense(X_check.shape[1], activation='sigmoid', name='output')(bottleneck)
autoencoder = Model(input_layer, output)

# Set encoder layer weights to all negative.
layer = autoencoder.layers[1]
weights = np.array([[-1, -1],[-1, -1], [-1, -1]])
biases = np.array([0, 0])
layer.set_weights([weights, biases])

# create encoder model.
encoder = Model(input_layer, bottleneck)

# create decoder model.
decoder_input = Input(shape=(2,), name='decoder_input')
decoder_layer = autoencoder.layers[-1]
decoder = Model(decoder_input, decoder_layer(decoder_input))

# Run auto-encoder, with [1, 0, 0], since encoder has all negative weights,
# and has 'relu' activation o/p of enocder should all be zeros. And that being
# the input of next sigmod we should get output [0.5, 0.5, 0.5]
output_data = autoencoder.predict(X_check)
print(output_data)
```


### Relevant log output

```shell
Python 3.8.17 (default, Jul  5 2023, 15:45:03) 
[Clang 14.0.6 ]
Python Platform: macOS-13.4-arm64-arm-64bit
Tensor Flow Version: 2.13.0
GPU is available
1/1 [==============================] - 0s 38ms/step
[[0.287966   0.85427195 0.28276426]]
```
