GPU VRAM usage significantly higher for LSTM models on TensorFlow v1.X when compared to other frameworks
Hi @jf9123 ,

Thanks for reporting. Is the reported issue is with TF1.x versions. Can it be reproducible with 2.x versions ? Since 1.x versions are not supported now this may not be considered for fix. Request you to try to reproduce the behaviour with 2.x Versions and let us know the results.Also request you to submit reproducible code. Thanks! @SuryanarayanaY Here's the code for reproducibility; https://github.com/jf9123/lstm_question/ Hi @jf9123,

I request you to please submit a colab gist as I can't able to reproduce the issue by cloning the repo. This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. @SuryanarayanaY The issue isn't present in TensorFlow v2.X. Was the behavior not observed on both TensorFlow v1.X and TensorFlow v2.X, or only TensorFlow v2.X? @jf9123 ,

Since support for TF1.x versions dropped our Team mostly will not look into the issue. I request if you can able to submit code snippet with TF2.x versions we will definitely review the issue. Please note that Keras integrated with Tensorflow only at Backend and if I am not wrong you are comparing keras 2.5v with TF 1.x versions right ?

I request you to compare with 2.x versions of TF and Keras and I am sure Tensorflow will be more performant since Keras is designed for user friendly and it uses Tensorflow only at backend from 2.x versions. 

Thanks! This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60495">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60495">No</a>
 @SuryanarayanaY I apologize for the confusion but the result reported was from TensorFlow 2.5. I just wrote and ran it without Keras.
### System information

-   **Have I written custom code (as opposed to using a stock example script
    provided in TensorFlow)**: Custom
-   **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: Linux Ubuntu 18.04
-   **TensorFlow installed from (source or binary)**: binary
-   **TensorFlow version**: 2.5
-   **Python version**: 3.7.5
-   **GPU model and memory**:  NVIDIA Quadro RTX 4000 GPU with 8GB of VRAM
-  **CPU model**: Intel(R) Core(TM) i9-9900K CPU operating at 3.60GHz 
- **cuDNN version**: 7.6.5


### Describe the problem
Hello everybody.

I’ve been experimenting with different models and different frameworks, and I’ve noticed that, when using GPU, GPU VRAM consumption when using LSTM models on the IMDB dataset is significantly higher on TensorFlow v1.X compared to the PyTorch and Keras implementations. Note that I artificially increased the size of the testing set of the MNIST dataset by duplicating the set by 100 times for a total of 1 000 000 samples. I've done so so any difference in inference time may be clearer.


Here are boxplots that showcase the GPU VRAM consumption of numerous LSTM models:

![Screenshot 2023-05-04 015324 - Copy (3)](https://user-images.githubusercontent.com/132307143/236126233-d175314f-70d0-461e-b0ed-cbe59b34533f.png)


Any ideas on what may be causing this?
