Initial training speed extreemly (~100x) slow for generated data, and become faster and faster as each epoch progress
Hi @mw66 
Apologies for the delay. In order to expedite the trouble shooting process, please share complete reproducible code. Thank you! This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60425">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60425">No</a>
 It's hard to add a test case, but just want to add it could be related to memory/disk swapping in a long running jib. So is there a way in Python before each epoch to ensure that trainX and trainY are in memory?
<details><summary>Click to expand!</summary> 
 
 ### Issue Type

Performance

### Have you reproduced the bug with TF nightly?

No

### Source

source

### Tensorflow Version

2.8.0

### Custom Code

No

### OS Platform and Distribution

Ubuntu 18.04.5 LTS

### Mobile device

_No response_

### Python version

3.9.1

### Bazel version

_No response_

### GCC/Compiler version

_No response_

### CUDA/cuDNN version

CUDA 11.1

### GPU model and memory

GeForce GTX TITAN X, 12G

### Current Behaviour?

My code is roughly like this:
```python
  for i in range(data_n):
    if not train_data[i].npz exists:
      data = generate train_data[i].npz and save to disk
    else:
      data = read train_data[i].npz from disk

    model.fit(data)
```

`data_n = 26`,  when `i` less than ~20, everything is normal.
as `i` reach about 20, something strange start to happen:

I found if the data.npz is generated, the initial training speed is *very* slow, e.g ~x100 slower than the normal speed (when directly read from .npz file), and as the training Epoch progress, it become faster and faster, until it eventually runs full power on GPU, e.g.
```
_________________________________________________________________                                                                                                                      
Epoch 1/1000                                                                                                                                                                           
732/732 [==============================] - 1524s 2s/step - loss: 3.0771 - accuracy: 0.2212 - val_loss: 3.0802 - val_accuracy: 0.2265                                                   
Epoch 2/1000                                                                                                                                                                           
732/732 [==============================] - 931s 1s/step - loss: 3.0538 - accuracy: 0.2246 - val_loss: 3.0623 - val_accuracy: 0.2303                                                    
Epoch 3/1000                                                                                                                                                                           
732/732 [==============================] - 833s 1s/step - loss: 3.0434 - accuracy: 0.2261 - val_loss: 3.0663 - val_accuracy: 0.2298                                                    
Epoch 4/1000                                                                                                                                                                           
732/732 [==============================] - 757s 1s/step - loss: 3.0339 - accuracy: 0.2275 - val_loss: 3.0618 - val_accuracy: 0.2309                                                    
Epoch 5/1000                                                                                                                                                                           
732/732 [==============================] - 663s 906ms/step - loss: 3.0316 - accuracy: 0.2279 - val_loss: 3.0692 - val_accuracy: 0.2297                                                 
Epoch 6/1000                                                                                                                                                                           
732/732 [==============================] - 572s 782ms/step - loss: 3.0254 - accuracy: 0.2288 - val_loss: 3.0683 - val_accuracy: 0.2298                                                 
Epoch 7/1000                                                                                                                                                                           
732/732 [==============================] - 406s 555ms/step - loss: 3.0201 - accuracy: 0.2291 - val_loss: 3.0638 - val_accuracy: 0.2305                                                 
Epoch 8/1000                                                                                                                                                                           
732/732 [==============================] - 367s 502ms/step - loss: 3.0148 - accuracy: 0.2301 - val_loss: 3.0595 - val_accuracy: 0.2289                                                 
Epoch 9/1000                                                                                                                                                                           
732/732 [==============================] - 321s 439ms/step - loss: 3.0098 - accuracy: 0.2311 - val_loss: 3.0599 - val_accuracy: 0.2299                                                 
Epoch 10/1000                                                                                                                                                                          
732/732 [==============================] - 304s 416ms/step - loss: 3.0058 - accuracy: 0.2318 - val_loss: 3.0608 - val_accuracy: 0.2301                                                 
Epoch 11/1000                                                                                                                                                                          
732/732 [==============================] - 286s 391ms/step - loss: 3.0053 - accuracy: 0.2317 - val_loss: 3.0562 - val_accuracy: 0.2320                                                 
Epoch 12/1000                                                                                                                                                                          
732/732 [==============================] - 268s 366ms/step - loss: 3.0010 - accuracy: 0.2315 - val_loss: 3.0563 - val_accuracy: 0.2323                                                 
Epoch 13/1000                                                                                                                                                                          
732/732 [==============================] - 252s 345ms/step - loss: 3.0003 - accuracy: 0.2320 - val_loss: 3.0518 - val_accuracy: 0.2332                                                 
Epoch 14/1000                                                                                                                                                                          
732/732 [==============================] - 233s 319ms/step - loss: 2.9963 - accuracy: 0.2321 - val_loss: 3.0488 - val_accuracy: 0.2315                                                 
Epoch 15/1000                                                                                                                                                                          
732/732 [==============================] - 217s 296ms/step - loss: 2.9926 - accuracy: 0.2326 - val_loss: 3.0515 - val_accuracy: 0.2329                                                 
Epoch 16/1000                                                                                                                                                                          
732/732 [==============================] - 198s 271ms/step - loss: 2.9881 - accuracy: 0.2341 - val_loss: 3.0514 - val_accuracy: 0.2327                                                 
Epoch 17/1000                                                                                                                                                                          
732/732 [==============================] - 181s 247ms/step - loss: 2.9883 - accuracy: 0.2326 - val_loss: 3.0527 - val_accuracy: 0.2333                                                 
Epoch 18/1000                                                                                                                                                                          
732/732 [==============================] - 165s 226ms/step - loss: 2.9868 - accuracy: 0.2338 - val_loss: 3.0496 - val_accuracy: 0.2307                                                 
Epoch 19/1000                                                                                                                                                                          
732/732 [==============================] - 153s 209ms/step - loss: 2.9831 - accuracy: 0.2333 - val_loss: 3.0438 - val_accuracy: 0.2336                                                 
Epoch 20/1000                                                                                                                                                                          
732/732 [==============================] - 137s 187ms/step - loss: 2.9817 - accuracy: 0.2338 - val_loss: 3.0442 - val_accuracy: 0.2330                    
Epoch 21/1000                                                                                                                                                                          
732/732 [==============================] - 123s 168ms/step - loss: 2.9779 - accuracy: 0.2340 - val_loss: 3.0434 - val_accuracy: 0.2344                                                 
Epoch 22/1000                                                                                                                                                                          
732/732 [==============================] - 113s 154ms/step - loss: 2.9789 - accuracy: 0.2344 - val_loss: 3.0370 - val_accuracy: 0.2341                                                 
Epoch 23/1000                                                                                                                                                                          
732/732 [==============================] - 102s 139ms/step - loss: 2.9743 - accuracy: 0.2351 - val_loss: 3.0392 - val_accuracy: 0.2356                                                 
Epoch 24/1000                                                                                                                                                                          
732/732 [==============================] - 93s 127ms/step - loss: 2.9777 - accuracy: 0.2349 - val_loss: 3.0517 - val_accuracy: 0.2336                                                  
Epoch 25/1000                                                                                                                                                                          
732/732 [==============================] - 82s 112ms/step - loss: 2.9716 - accuracy: 0.2352 - val_loss: 3.0279 - val_accuracy: 0.2369                                                  
Epoch 26/1000                                                                                                                                                                          
732/732 [==============================] - 79s 107ms/step - loss: 2.9709 - accuracy: 0.2351 - val_loss: 3.0363 - val_accuracy: 0.2348                                                  
Epoch 27/1000                                                                                                                                                                          
732/732 [==============================] - 67s 92ms/step - loss: 2.9672 - accuracy: 0.2354 - val_loss: 3.0344 - val_accuracy: 0.2343                                                   
Epoch 28/1000                                                                                                                                                                          
732/732 [==============================] - 55s 75ms/step - loss: 2.9664 - accuracy: 0.2350 - val_loss: 3.0418 - val_accuracy: 0.2337                                                   
Epoch 29/1000                                                                                                                                                                          
732/732 [==============================] - 45s 62ms/step - loss: 2.9690 - accuracy: 0.2356 - val_loss: 3.0439 - val_accuracy: 0.2340                                                   
Epoch 30/1000                                                                                                                                                                          
732/732 [==============================] - 32s 44ms/step - loss: 2.9639 - accuracy: 0.2364 - val_loss: 3.0333 - val_accuracy: 0.2350                                                   
Epoch 31/1000                                                                                                                                                                          
732/732 [==============================] - 28s 39ms/step - loss: 2.9631 - accuracy: 0.2362 - val_loss: 3.0373 - val_accuracy: 0.2353                                                   
Epoch 32/1000                                                                                                                                                                          
732/732 [==============================] - 28s 38ms/step - loss: 2.9626 - accuracy: 0.2368 - val_loss: 3.0391 - val_accuracy: 0.2343                                                   
Epoch 33/1000                                                                                                                                                                          
732/732 [==============================] - 26s 36ms/step - loss: 2.9598 - accuracy: 0.2355 - val_loss: 3.0339 - val_accuracy: 0.2356                                                   
Epoch 34/1000                                                                                                                                                                          
732/732 [==============================] - 26s 35ms/step - loss: 2.9581 - accuracy: 0.2368 - val_loss: 3.0337 - val_accuracy: 0.2349                                                   
Epoch 35/1000                                                                                                                                                                          
732/732 [==============================] - 25s 35ms/step - loss: 2.9601 - accuracy: 0.2366 - val_loss: 3.0306 - val_accuracy: 0.2363                            
```

As you can see, the 1st epoch took 1524s, while the 35th epoch took 25s; if the data is directly read from .npz an epoch took consistent ~18s.

Could this be a TF memory issue ? or it's a Python issue?

How and where I can debug such issue? or any suggestions I can try to improve the speed?

Thanks.

### Standalone code to reproduce the issue

```python
# My code is roughly like this:

  for i in range(data_n):
    if not train_data[i].npz exists:
      data = generate train_data[i].npz and save to disk
    else:
      data = read train_data[i].npz from disk

    model.fit(data)
```
```


### Relevant log output

_No response_</details>