Python 3.11 TF memory leaks
@sachinprasadhs @SuryanarayanaY
Any news on this issue?  I'm having same issue. @0x0L have you fixed this? @qiangpei I tried using tcmalloc but that didn't change a thing... 

I am switching to torch for my next projects 
### Issue type

Bug

### Have you reproduced the bug with TensorFlow Nightly?

Yes

### Source

source

### TensorFlow version

2.15

### Custom code

Yes

### OS platform and distribution

Debian GNU/Linux 11 (bullseye) 

### Mobile device

_No response_

### Python version

3.11

### Bazel version

_No response_

### GCC/compiler version

_No response_

### CUDA/cuDNN version

_No response_

### GPU model and memory

_No response_

### Current behavior?

Basic tensorflow calls leak memory in python 3.11

Running in basic docker envs:
```Dockerfile
# FROM python:3.10.11
FROM python:3.11.2

RUN pip install psutil tensorflow
# RUN pip install psutil tf-nightly
```

The following code (which only runs `tf.zeros(shape=(1,))` in a loop !!!)
```python
import sys
import time
print("PYTHON VERSION", sys.version)

import psutil
process = psutil.Process()

import tensorflow as tf
print("TF VERSION", tf.__version__)

def bench(fn):
  t = time.monotonic()
  r = []
  for i in range(100):
      for j in range(5000):
          fn()
  
      mem = process.memory_info().rss / 1024 ** 2
      r.append(mem)
  return r[-1] - r[0], time.monotonic() - t


def fn():
    return tf.zeros(shape=(1,))

tf_fn = tf.function(fn)

if __name__ == "__main__":
    fn()  # warmup
    tf_fn()

    mem_usage, elapsed = bench(fn)
    print(f"EAGER memory growth {mem_usage:.1f} MB in {elapsed:.1f} s")

    mem_usage, elapsed = bench(tf_fn)
    print(f"TF FUNC memory growth {mem_usage:.1f} MB in {elapsed:.1f} s")
```
produces the following matrix of results:

```
PYTHON VERSION 3.10.11 (main, May 23 2023, 13:58:30) [GCC 10.2.1 20210110]
TF VERSION 2.15.0
EAGER memory growth 0.0 MB in 47.9 s
TF FUNC memory growth 0.0 MB in 149.4 s
```

```
PYTHON VERSION 3.10.11 (main, May 23 2023, 13:58:30) [GCC 10.2.1 20210110]
TF VERSION 2.16.0-dev20231203
EAGER memory growth 0.0 MB in 47.1 s
TF FUNC memory growth 0.0 MB in 154.1 s
```

```
PYTHON VERSION 3.11.2 (main, Mar 23 2023, 17:12:29) [GCC 10.2.1 20210110]
TF VERSION 2.15.0
EAGER memory growth 45.6 MB in 46.2 s
TF FUNC memory growth 15.2 MB in 154.4 s
```

```
PYTHON VERSION 3.11.2 (main, Mar 23 2023, 17:12:29) [GCC 10.2.1 20210110]
TF VERSION 2.16.0-dev20231203
EAGER memory growth 45.6 MB in 46.8 s
TF FUNC memory growth 15.2 MB in 154.9 s
```

The issue has been there since the first available 3.11 version (tf 2.12rc0 if I remember correctly)


### Standalone code to reproduce the issue

```shell
-
```


### Relevant log output

_No response_