rnn with initial_state model can't be loaded with load_model 
@xing-w I was able to replicate the issue reported [here](https://colab.research.google.com/gist/sushreebarsa/df202f7ea6ad3c85bdf4184cc8e1c9a1/rnn_save_model.ipynb#scrollTo=qHLU9PbnkU4g). 
This issue seems to be Keras issue.  Please post this issue on [keras-team/keras repo.](https://github.com/keras-team/keras/issues) as Keras development is fully moving to [github.com/keras-team/keras](http://github.com/keras-team/keras). All issues and PRs related to keras will be addressed in that repo.
To know more see this TF forum discussion ; 
[https://discuss.tensorflow.org/t/keras-project-moved-to-new-repository-in-https-github-com-keras-team-keras/1999](https://discuss.tensorflow.org/t/keras-project-moved-to-new-repository-in-https-github-com-keras-team-keras/1999)
Thank you! Posted it to keras. Thanks. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61732">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61732">No</a>

### Issue type

Bug

### Have you reproduced the bug with TensorFlow Nightly?

Yes

### Source

source

### TensorFlow version

2.13.0

### Custom code

Yes

### OS platform and distribution

_No response_

### Mobile device

_No response_

### Python version

_No response_

### Bazel version

_No response_

### GCC/compiler version

_No response_

### CUDA/cuDNN version

_No response_

### GPU model and memory

_No response_

### Current behavior?

A simple RNN with LSTMcell model.
I want to initialize the states with `initial_state_h` and `initial_state_c`. 
```
batch_size= 16
inputs = tf.keras.layers.Input(shape=(20,5),batch_size=batch_size)
units = 8
lstm_cell_fw = tf.keras.layers.LSTMCell(units)

initial_state_h = tf.random.normal(shape = (batch_size,units), mean=0., stddev=10., dtype=tf.dtypes.float32)
initial_state_c = tf.random.normal(shape = (batch_size,units), mean=0., stddev=10., dtype=tf.dtypes.float32)
lstm_layer_fw = tf.keras.layers.RNN(lstm_cell_fw, stateful=True, return_state=True, return_sequences=False)
outputs,states_h_fw, states_c_fw= lstm_layer_fw(inputs,initial_state = [initial_state_h,initial_state_c])

lstm_dense1 = tf.keras.layers.Dense(16, activation = 'relu')
lstm_dense2 = tf.keras.layers.Dense(2, activation = 'softmax')
out=lstm_dense2(lstm_dense1(outputs))

model = tf.keras.models.Model(inputs, out)
```
After compile and train, the model is saved with `model.save('my_model_test.keras')`.

```
model.compile(optimizer='adam', loss='categorical_crossentropy',metrics=['accuracy'])
model.summary()

xTrain = np.random.rand(96,20,5)
yTrain = np.random.rand(96,2)

for i in range(10):
  model.fit(xTrain, yTrain,batch_size=batch_size)

model.save('my_model_test.keras')

```
But when I try to load it with `load_model = tf.keras.models.load_model('my_model_test.keras')`, it gives error:
```
13 frames
[/usr/local/lib/python3.10/dist-packages/keras/src/backend.py](https://localhost:8080/#) in int_shape(x)
   1530     """
   1531     try:
-> 1532         shape = x.shape
   1533         if not isinstance(shape, tuple):
   1534             shape = tuple(shape.as_list())

AttributeError: 'float' object has no attribute 'shape'

```
I tried to save in other format, `.h5`, `.json`, etc. All give the same error.

But, if I don't use `initial_state` in `outputs,states_h_fw, states_c_fw= lstm_layer_fw(inputs)`, everything goes well. No problem with `load_model`.

### Standalone code to reproduce the issue

```shell
https://colab.research.google.com/drive/1uKEpnddzeYSRG_1vKjtcQ4OLNwLuQeqy?usp=sharing
```


### Relevant log output

```shell
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-7-8e0130abf25e> in <cell line: 1>()
----> 1 load_model = tf.keras.models.load_model('my_model_test.keras')

13 frames
/usr/local/lib/python3.10/dist-packages/keras/src/backend.py in int_shape(x)
   1530     """
   1531     try:
-> 1532         shape = x.shape
   1533         if not isinstance(shape, tuple):
   1534             shape = tuple(shape.as_list())

AttributeError: 'float' object has no attribute 'shape'
```
```
