Memory leak using SavedModel format
Possibility 
https://github.com/keras-team/keras/issues/18419 @innat,
Could you please share a reproducible code that supports your statement so that the issue can be easily understood? Thank you! Try with:
1. Building the model and using model.save(my_model) instead of model.save_weights(my_model.h5) to save the entire model.
2. Loading the model using tf.keras.models.load_model(my_model) without passing jit_compile=True.
3. Initializing the model outside of the strategy.scope() block before saving the weights to avoid the OSError. @tilakrayal It might take some time to create reproducibility code as the underlying issue hard to interpret. In the meantime, if there is any info/doc that I could take a look, please share. 

@msf-caesar Thanks for the suggestions.

1. I could use only `model.save` but it takes lots of memory (as described above) compared to manually load weights. Also, in my pipeline, I like to add both way to make it feature complete. 
2. I could do that, but why should I skip passing `jit_compile`? If a model is not compatible and still this params is passed, I think the API should handle it internally, maybe with warning message that the model is not jit compatible and running without it. But the main discrepancy is that, I could pass jit compile (and get the effect) while manually loading the weight to the model. I guess, `model.save` API did something internally which causing this issue. 
3. Again, whether initializing the model in scope or out of it, both model should build and can be saved. This matter only if all components of a model is not initialzing the same (in scope or not). Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61753">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61753">No</a>

### Issue type

Bug

### Have you reproduced the bug with TensorFlow Nightly?

Yes

### Source

source

### TensorFlow version

2.12 / 2.13

### Custom code

No

### OS platform and distribution

_No response_

### Mobile device

_No response_

### Python version

_No response_

### Bazel version

_No response_

### GCC/compiler version

_No response_

### CUDA/cuDNN version

_No response_

### GPU model and memory

_No response_

### Current behavior?

Simply, I did

- Build the model (`model`)
- Save it using (a). `model.save(my_model)` , (b). `model.save_weights(my_model.h5)`.

Next, 


## Case 1

1. Using `model.load_weight(my_model.h5)`, and pass `jit_compile=True` followed by `model.fit`. It works, though 11GB consumed out of 16GB. 
2. Using `tf.keras.load_model(``my_model`), and pass `jit_compile=True`, followed by `model.fit`. It doesn't work, simply extremly slow and consumed 15.8GB instantly. Sudden jump from 3GB to 15.8GB. But without passing `jit_compile`, it runs normally but memory consumption remains same. 

## Case 2

1. If I initialize model within `with strategy.scope():`, and try to save the weight (by `model.save_weight`), it gives `OSError: Unable to create link (name already exists)` but without strategy scope, this error doesn't occur.

### Standalone code to reproduce the issue

```shell
to do.
```


### Relevant log output

_No response_